Usage KPIs Surpass Most Business KPIs

Recently, we were talking about the interest of User Stories to bring users at the center of the product development strategy, and to ensure developers always work in the interest of the user.

Today, let’s introduce another benefit of the user stories: the indicators and insights they provide for your product success.

We call them Usage KPIs.

Yet another layer of indicators?

Every online business owner is eager for Key Performance Indicators (KPIs) that measure their performance. These indicators are statements about the company health, that allow managers and employees to bench and understand what goes right or wrong, in their action, and lead to make decisions.

In 2017, many tools can serve business owners hundreds of KPIs. So much that, it can get difficult to choose the right ones, that may lead to misjudgment, contradictory positions or dispersion.

As we are about to introduce a new layer of indicators, it is fair to ask ourselves :

If we need new indicators?
How to organize your structure around and cope with different types of indicators
Which ones are essential, or not?

The need for indicators closer to reality

It is interesting to see how a leader addresses this question.

Blablacar.com is one of the highest valued startup in Europe, and a proven example of success in every field: 1B€+ valuation, millions of people transported each month, HappyAtWork’s 2015 award. Dealing on a daily basis with 400+ employees and millions of users is a thing. Better choose carefully the indicators that bench performance, measure teams’ and individuals’ accomplishments, and the ones to communicate on.

As a data-driven company, Key Performances Indicators (KPIs) are at the forefront of all activities at Blablacar – Fred Mazzella

KPIs are so important in Blablacar’s strategy, at every level of the company, that they eventually became one of their 10 company key values:

Because data remain blind figures if you don’t add interpretation and meaning behind, it is interesting to look at how Blablacar distinguished three kinds of indicators.

Vanity indicators : the figures which serve their communication. For instance, the total number of members registered on their platform.
Sanity indicators : the figures which observe their performance. They are less talkative with this section but you can easily imagine them: ARPU, acquisition cost, …
Reality indicators : the figures which translate what is actually/really happening with the service on a daily basis : the given examples are about the number of rides conducted or people transported every day.

By applying these “labels”, they facilitated the access and the understanding of the information for everyone in the company. Each data has a purpose, and a context.

In order to grow, the team is encouraged to look at the right metrics, and improve them. – Fred Mazzella

Blablacar is high-in-the-sky now, but they needed ground-level indicators to stay in contact with the real world, and give to every of their employees a control on their activity.

As Blablacar do, every business should find, select and organise the indicators that matters to their own activity.

Business KPIs vs. Usage KPIs

For most of us who don’t have 35 millions users yet, the two main business KPIs followed – apart from financial (revenue, margin, average revenue per user & lifetime value .. ) – are commonly the conversion & retention figures.

Conversion, or Traction, is (almost) everything for a starting project. Because you can spend months building the service you like, the truth remains that it will become a product only once you have a growing audience starting using it. Without users, it worths only the love and energy you put into it.
Retention is (almost) all the rest. Once you convinced someone to use your service, being unable to keep him engaged long enough is another loss of energy.

Tracking conversion & retention is how we validate that we actually have a business running, and furthermore, a viable one that brings money home.

However, both Conversion & Retention are the result of what happens at a deeper level in the product usage. And delivering performance in the product usage is directly linked with the user stories you defined earlier.

From User Stories to Usage KPIs

When you define User Stories for a feature, you write down both the expected outcome (goal, or action accomplished by the user) of this increment, and the value you (expect to) deliver.

Then how to be sure that your users are actually doing what you expect them to do?

Usage KPIs measure your User’s ability to actually use the service you deliver.

If you defined user stories, this task becomes much more easy. As you do for each of your conversion funnels, you can now track the actions performed, and check if they fill in the pattern you defined, and match the value you expressed.

Let’s quickly draw how it could like for one of our features – the example we previously defined:

As a User of Onbrowse.io’s session replay feature,

I can review every visitor session as if I were behind their screen,

so that I can understand what goes wrong in their journey (and improve it).

Here are the questions we have to ask ourselves are, and the KPIs we can define for it :

who entered the session replay feature, or not > share of users / non-users
what is the context behind of the use of this feature? what figure was they looking before to get to look into sessions replay > entry page, history
where they looking for a specific behavior / use case > filters used, which ones
how many sessions did they replayed before to get to a conclusion > number of sessions observed
did they actually find interesting cases in the sessions they viewed that allow them to understand some unexpected behavior > number of starred / shared sessions

These indicators give you a real understanding on each increment of your product.

Their benefit goes beyond the product strategy. We aforesaid the correlation and consequence link between usage and conversion / retention. Define Usage KPIs for every feature you deliver, in order to measure their impact on:

Conversion : understand why people get onboard of your service or not, and the importance of each feature on this decision.
Retention : detect where your users struggle with your service in the use, or where the value you deliver is just not sufficient to keep them onboard

This micro-management for each of your features validate that :

you went for the right product decisions: the user stories you defined match with a need and behavior which really exists
you went for the right execution: the product is usable in the manner you developed it
eventually, you effectively solve the problem of the people in your target

Usage KPIs: nice to have, or must-have?

One simple fact:

If your product is not usable – if your users (the ones you convinced to use your service) are not able to do whatever you want them to achieve with it – then they will leave you and turn to your first competitor.

Knowing this, for each of your User Stories you must be able to :

Measure what share of the users actually are able to perform the action.
Know what is the gap between the users that were and were not able to perform it.
Understand how people were able to perform the action properly.
Understand what happened for people who were not able to perform it.

Knowing this is the only way to ensure that you deliver the expected value to your users.