Seven Lessons from Running 22,000 AB Tests

The average success rate of most AB testing programs hovers around 12%. Yep, for every 100 experiments you run, only 12 of them will generate an increase in conversions. That is worse than a coin toss. Too many companies start AB testing hoping that it will deliver a great increase in conversions. A few months later, they are bitterly disappointed.

In this session, I will share the lessons we learned by analyzing over 22,000 AB tests on 700 CRO projects. These lessons paint the common patterns that helped us deliver upwards of 50% success rates on AB testing programs. The lessons provide a strategic roadmap for a CRO project as well as tactical details needed while running the day-to-day operations.

After this session, you’ll be able to:

  • Understand what it really takes to run a successful CRO program (Time, resource, management, etc.)
  • Find the right balance between strategy and tactics when running an experimentation program
  • See an actual project management board running a CRO program