Understanding Product Metrics: A Guide to Effective Measurement
Written on
Chapter 1: The Role of Metrics in Tech Companies
Metrics are essential for businesses to evaluate how effectively a product or feature achieves specific goals. These metrics can range from straightforward figures like total profit over a set timeframe to more complex calculations such as customer acquisition costs or conversion rates, all contributing, either directly or indirectly, to overall profitability.
This article will concentrate on the metrics utilized by B2C tech firms to assess and enhance engagement through experimentation, using Spotify as a primary example.
Section 1.1: Defining the North Star Metric
Establishing what is truly significant is often referred to as determining the 'north star metric.' Enhancing this metric typically results in measurable benefits for the organization.
Consider working within Spotify’s music recommendation team. If asked what is crucial for Spotify, you might suggest the total number of songs played or the total listening duration. Rather than deeming either answer incorrect, it's more accurate to say that both perspectives hold merit. An increase in the total songs played will likely lead to a rise in total listening duration, given that the average song length remains constant.
Total listening duration can be calculated as follows:
Total duration of music played = Total number of songs x Average duration of a song
However, the challenge lies in understanding user behavior on Spotify. Some users may explore new tracks and switch songs within seconds if they don’t enjoy them. Not every listener stays until the end of a song; many may switch midway.
Thus, it’s vital to define success criteria for when a song is considered 'played' by a user. Although this definition may seem subjective, establishing a clear baseline—say, 30 seconds—allows for effective tracking. Since users may listen to varying amounts of music in a single session, averaging this figure at the individual level yields a more insightful metric.
A more fitting north star metric could be articulated as:
Songs (successfully) played per user = Average number of songs played (for 30 seconds or more) / Total number of users
Section 1.2: Identifying Influencing Factors
Next, we’ll explore various elements that can impact these metrics. For instance, when analyzing daily statistics, consider:
- The average number of songs users view on the platform (views per user)
- The percentage of songs that users opt to play (click conversion rate or clicks per view)
- The percentage of songs that are successfully played (plays per click)
Bringing these together will yield our north star metric:
Songs played per user = views per user x clicks per view x plays per click
Enhancing each of these figures will directly affect the overall metric.
Note: Metrics can be interpreted in various ways and should be derived accordingly. For example, within the Spotify interface, users can jump to the next track by clicking the 'next' button or swiping right. If a user clicks ‘next,’ listens to a song for just 5 seconds, and then clicks again, how do you classify that song? Is it viewed, clicked, both, or neither?
Chapter 2: Diving Deeper into Feature-Specific Metrics
Despite the metrics we’ve examined, they may not provide enough insight or actionable data across all of Spotify. The platform is renowned for its personalized song recommendations, allowing users to create their own playlists, which inherently have a better performance in metrics due to user preference.
In other words, self-curated playlists could yield better efficiency (clicks per view x plays per click), warranting a separate evaluation of metrics at the individual feature level to facilitate informed hypotheses.
For instance, you might assess a new recommendation algorithm based solely on its effectiveness for suggested songs. But what if this leads to a decline in metrics for self-created playlists? Does that signify limited overall value? How can insights into business priorities guide such decisions?
The example above encourages a metrics-oriented mindset. As illustrated, metrics are subjective and depend on how they are defined and utilized. A clearer comprehension of their application and purpose, alongside an awareness of business implications and user perspectives, is vital.
You can further challenge your understanding by attempting to define metrics for various companies like Amazon, YouTube, Tinder, and Medium.com. You may find surprising similarities.
Thank you for engaging with this article; I look forward to exploring the world of metrics more deeply in the future!
This video titled "The Secret of Product Metrics" provides valuable insights into understanding product metrics and their application within tech companies.
In the video "Product Metrics: How to Measure Product Success," viewers will learn effective strategies to measure product success through various metrics.