Let’s look at 4 reasons why you should use the One Metric That Matters.
- It answers the most important question you have.
- It forces you to draw a line in the sand and have clear goals.
- If focuses the entire company.
- It inspires a culture of experimentation.
To succeed at that, you need to actively encourage experimentation. It will lead to small-f failures, but you can’t punish that. Quite the opposite: failure that comes from planned, methodical testing is simply how you learn.
Randy explained when staffing costs exceed 30% of gross revenues, that’s bad, because it means that you’re either spending too much on staff or not deriving enough revenue per customer.
But not all people are equal. The plain truth is that not every user is good for you.
Not all customers are good. Don’t fall victim to customer counting. Instead, optimize for good customers and segment your activities based on the kinds of customer those activities attract.
By mid-2008, Circle of Friends had 10M users. Mike focused on growth above anything else. “It was a land grab,” he says, and Circle of Friends was clearly viral. But there was a problem. Too few people were actually using the product.
According to Mike, less than 20% of circles had any activity whatsoever after their initial creation.
Changing the company’s CTA from “Get started free” to “Try it out free” increased the number of people who clicked on an offer by 376% for a 10-day period.
A/B tests seem relatively simple, but they have a problem. Unless you’re a huge web property — like Bing or Google — with enough traffic to run a test on a single factor like link color or page speed and get an answer quickly, you’ll have more things to test than you have traffic.
We had the intuition that Mac users are 40% more likely to book a 4- or 5-star hotel than PC users and to stay in more expensive rooms, and we were able to confirm it based on the data.
Humans do inspiration; machines do validation.
Optimization is all about finding the lowest or highest values of a particular function. A machine can find the optimal settings for something, but only within the constraints and problem space of which it’s aware, in much the same way that the water in a mountainside lake can’t find the lowest possible value, just the lowest value within the constraints provided.
Evolution, he explains, can create the eye. In fact, it can create dozens of versions of it, for wasps, octopods, humans, eagles, and whales. What it can’t do well is go backward: once you have an eye that’s useful, slight mutations don’t usually yield improvements.
- Assuming the data is clean. Cleaning the data you capture is often most of the work, and the simple act of cleaning it up can often reveal important patterns.
- Not normalizing.
- Excluding outliers. Those 21 people using your product more than a thousand times a day are either your biggest fans, or bots crawling your site for content. Whichever they are, ignoring them would be a mistake.
- Including outliers.
- Ignoring seasonality.
- Ignoring size when reporting growth.
- Data vomit. A dashboard isn’t much use if you don’t know where to look.
- Metrics that cry wolf.
- The “Not Collected Here” syndrome.
Some entrepreneurs are manically, almost compulsively, data-obsessed, but tend to get mired in analysis paralysis. Others are casual, shoot-from-the-hip intuitionists who ignore data unless it suits them, and pivot lazily from idea to idea without discipline. At the root of this divide is the fundamental challenge that Lean Startups advocates face: how do you have a MVP and a hugely compelling vision at the same time?
Plenty of founders use Lean Startup as an excuse to start a company without a vision.
What’s worse, as a founder and entrepreneur, you have strong, almost overwhelming preconception about how other people think, and these color your decisions in subtle and insidious ways.
Analytics can help. Measuring something makes you accountable. You’re forced to confront inconvenient truths. And you don’t spend your life and your money building something nobody wants.
We’re all delusional — some more than others. Entrepreneurs are the most delusional of all.
Entrepreneurs are particularly good at lying to themselves. Lying may even be a prerequisite for succeeding as an entrepreneur — after al, you need to convince others that something is true in the absence of good, hard evidence. You need believers to take a leap of faith with you. As an entrepreneur, you need to live in a semi-delusional state just to survive the inevitable rollercoaster ride of running your startup.
A good metric changes the way you behave. This is by far the most important criterion for a metric: what will you do differently based on changes in the metric?
Vanity metrics might make you feel good, but they don’t change how you act.
Leading metrics give you a predictive understanding of the future; lagging metrics explain the past. Leading metrics are better because you still have time to act on them — the horse hasn’t left the barn yet.
Analysts look at specific metrics that drive the business, called KPIs.
Qualitative data is messy, subjective, and imprecise. It’s the stuff of interviews and debates. It’s hard to quantify. You can’t measure qualitative data easily. If quantitative data answers “what” and “how much,” qualitative data answers “why.” Quantitative data abhors emotion; qualitative data marinates in it.
Whenever you look at a metric, ask yourself, “What will I do differently based on this information?” If you can’t answer that question, you probably shouldn’t worry about the metrics too much.
Just a few years ago, many analysts and investors were wondering whether social media was going to lead to the end of email. In an ironic twist of fate, it turns out that social media adoption is driven by email. More and more social application are leveraging the power of email to drive repeat usage and retention.
Phone companies devote considerable effort to tackling this kind of churn. They build sophisticated models that predict when a subscriber is about to cancel her service, and then offer her a new phone or a discount on a renewed contract just before the cancellation happens.
There’s a natural progression of metrics that matter for a business that change over time as the business evolves. The metrics start by tracking questions like “Does anyone care about this at all?” and then get more sophisticated, asking questions like “Can this business actually scale?”
Those early adopters will be vocal, but beware. Their needs might not reflect those of the bigger, more lucrative mainstream.
We kept hearing over and over that monthly subscriptions were the key to growing as a successful SaaS business. So that’s the direction we took, but it didn’t work as planned.
For mobile developers, the dynamics of an app store matter more than almost anything else when it comes to achieving significant traction. Being showcased on the homepage of the App Store routinely yields a hundredfold increase in traffic.