Why Most of Your Experiments Needs to Fail?
And your team should embrace high "failure rate".
If your experiments mostly succeed, you're not experimenting, you're just validating what you already know. Growth isn’t about getting it right every time; it’s about learning. The best growth teams know this: failure is not just a risk, it’s a necessity. Every failure is a lesson, a data point that brings you closer to the breakthrough.
In this article, I want to talk about why you and your team constantly need to fail to build a great A/B testing program and more importantly, get the mindset right. It might sound easy, but for 10 years of working with various teams across multiple startups, maybe I`ve seen very few teams get it right.
Let’s dive in.
Why You Need to Fail A Lot in A/B Testing?
Growth is about challenging the status quo, and to do that effectively, you need to take ambitious and uncertain bets. Most of these bets will not work out - and that’s okay. Most experiments should fail because the purpose of experimentation is to learn. If you’re getting a high success rate, you’re likely not pushing the boundaries enough.
Imagine aiming only for what feels safe. You might end up with small wins, but you’ll never find those massive, game-changing breakthroughs that truly drive exponential growth for your startup. Failure is a signal that you’re reaching beyond the obvious, looking for the insights that everyone else is missing.
Experimentation: Learning vs. Validation
The role of experiments is not to prove what you already know; it’s to uncover what you don’t know. Startups live and die by their ability to iterate quickly and discover what works. If your experiments always succeed, then you’re not growing, you’re just validating.
In a high-growth environment, you need to continuously test and iterate. Not everything you try will work, and that’s the point.
For instance, if you test a new onboarding flow and it doesn’t lead to reduced drop-off, that’s critical information. Maybe it means that your users need more up-front value or a different kind of nudge. Either way, you’ve learned something new that can now inform your next iteration.
Failure Proves You're Pushing Boundaries
Growth doesn’t live in the comfort zone. If you’re succeeding too often, it’s a clear signal that you’re staying in your comfort zone and not pushing enough. Real growth lies beyond what’s obvious and proven.
Look at Amazon’s Fire Phone—a huge failure by most conventional standards. But it’s this kind of moonshot thinking that ultimately gave rise to products like Alexa and Echo, built upon the lessons learned from those past mistakes.
Read the - The Story Behind the Amazon Fire Phone Failure.
Calculating risks and learning from mistakes are what set the stage for success. Each failure adds to your team’s collective knowledge, making future experiments sharper and more informed. If your team isn't failing, they aren't testing the limits hard enough to uncover the next big growth lever.
Winning Is Not the Goal - Learning Is
Every failure generates a new data point. Each failed experiment is a stepping stone to understanding your users better, refining your strategies, and discovering what drives engagement or conversion. The goal is to fail fast, learn fast, and adapt. That’s where real growth comes from.
A strong hypothesis is not just about proving you’re right. It's about what happens if you're wrong - because that's when real learning takes place. Every failure forces you to challenge your assumptions and understand your blind spots. That's how you find the gaps in your user journey, your product, or even in the market.
Creating a Culture That Embraces Failure
Teams that drive consistent growth are those that have normalized failure. If your team is afraid of getting it wrong, they’ll play it safe. They’ll aim for easy wins and miss the truly transformative ideas. This culture of safety is a killer for real growth.
Leadership needs to champion failure as a key part of the growth process. When leaders celebrate experiments that didn’t go well but taught valuable lessons, it signals to the entire organization that the goal isn’t just to win—it’s to learn. Netflix is a prime example of this. They run countless tests each year, and most of them fail. But it's those failures that lead to product improvements that hook users.
Read - Netflix: A Culture of Learning
If your team fears failure, the result is incremental improvements that don’t push the needle. But by reframing failure as an opportunity, you unlock creativity. The bold ideas, the ones that carry significant risk, are often the ones with the potential for transformative impact. To innovate, you must make room for failure.
Practical Steps to Embrace Failure in Growth
To fully embrace a growth mindset that values failure as part of the process, here are structured steps, supported by examples and metrics from successful experimentation programs across different industries.
1. Set Ambitious, Risky Goals
Top experimentation teams know that the most ambitious goals carry a high risk of failure, and they lean into that risk. Data from leading growth programs show that teams with a failure rate between 60% and 80% discover the biggest wins.
For instance, Google’s experimentation programs report a success rate of only about 10-20%, meaning 80-90% of their experiments do not meet their goals, but the learnings from each attempt build a foundation for future success.
Your goals should be so ambitious that some level of failure is likely. If everything is succeeding, then your targets are too conservative. Instead of aiming for small, incremental boosts, aim to double or triple key metrics. The result? Even if you fail, the process will yield valuable insights, and you may still achieve impressive gains.
2. Track Learnings, Not Just Wins
High-performing teams dedicate resources to learning tracking systems. A notable example is Booking.com, which runs over 25,000 experiments annually, each one meticulously documented whether it succeeds or fails. This documentation process enables them to create a rich “Failure Playbook” that’s accessible across teams, ensuring that every experiment contributes to the collective intelligence of the organization.
Every experiment should end with a documented insight. Create a system where both successes and failures are reviewed with the same level of scrutiny. Make a “Failure Playbook” - a resource that your entire team can use to understand what hasn’t worked, and why.
3. Celebrate Failures and the Learnings They Bring
Companies like Microsoft and Slack have adopted a structured approach to celebrating failures. At Microsoft, teams hold a monthly “Failure Review” where key failed experiments are dissected, and the most insightful lessons are showcased. Slack has “Lessons Learned” sessions that highlight both success and failure stories, cultivating a space where experimentation is viewed as a continual learning process.
Hold retrospectives specifically focused on failed experiments and what they taught you. Celebrate the most insightful failures, and make sure the whole team benefits from the learnings. At Slack, for example, they share “Lessons Learned” after experiments, which encourages open conversations about what didn’t work and why.
4. Iterate Quickly and Build on Failures
Speed in iteration is crucial to maximizing learning. Amazon’s rapid prototyping approach is a great example - they often take an experimental failure, implement changes, and re-test within a matter of weeks. Their “Two-Pizza Team” structure helps keep these iteration cycles efficient and cross-functional, driving more value out of every failed experiment.
Speed is your friend. If an experiment fails, take what you’ve learned and feed it directly into the next iteration. The faster you can iterate, the more you’ll learn. Lightweight experiments and prototyping reduce the cost of failure and increase the speed of learning.
In conclusion, I would say that if you’re not failing, you’re not growing. Real growth happens at the edge of your comfort zone, where you’re taking bets and pushing the boundaries of what you know. Systematic experimentation, with failure as an expected outcome, is what differentiates good teams from great ones.
Failure is not the opposite of success - it’s an essential component of the path to success. The more you fail, the more you learn.