- Book: Lean Analytics Book: The Effective Engineer
Good metrics are important
- They should be comparable across time
- Ideally, they should be rates (i.e. have a time dimension)
Metrics should be actionable
- They help you make better decision faster
- Ex: Percentage of active users (Your actions can cause it to change)
- Another Ex: Customers acquired over a certain time period
- Basically we want to feel the direction as much as the velocity.
- Negative Example: Total number of users (It will always be growing so hard to get a feel for the direction)
For the same reason, it’s always preferable to have lead indicators vs lag indicators
Ensure that metrics measure the right thing.
Make sure you set up metrics for your known knowns.
- This includes things you know you want to measure.
To tackle the unknown unknowns, it’s important to routinely dive through metrics and explore
In metrics, correlation is good but causation is better
- Causative metrics are actionable.
Metrics are a line in the sand
- It’s important to reflect and update our metrics.
- Exploring the data helps with this.
Questions to keep in mind about metrics.
To prevent regressing metrics, one technique is to use metric ratcheting