What is statistical significance?

Haven't set up an A/B test yet? Check out our guide to help get you started.

When you set up an A/B test, you'll see the following:343434343434.png

You may have noticed a number called "Statistical Significance". But what does this number mean?

To keep it basic, statistical significance indicates how accurate your A/B test results are.

We highly recommend running an A/B test until you reach 95%+ statistical significance to get accurate test results. How long this takes will depend on:

  • How many variants you're testing
  • How much traffic you have seeing the test campaign
  • The conversion rate of your baseline (lower = slower significance increase)
  • The improvement of variants vs. the baseline (higher improvement = faster significance increase)
Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.