Experiment Design

From Hypothesis to Experiment Design

Before running an experiment, it is important to outline what hypothesis you want to test. This could mean running a single experiment or it could mean breaking down the hypothesis into multiple experiments. For e.g. if your hypothesis is that there may be higher willingness to pay for your products in North America, you may want to break down the hypothesis into two experiments - one experiment across US and Canada (since they are likely similar markets for your business), and another for Mexico.

If your hypothesis is a little more specific - say, there is higher willingness to pay amongst professionals in North America, it is best to define the relevant audience on the Corrily dashboard, and segment your experiment further. However, if you do not have a strong hypothesis, we recommend starting broad, learning from the experiment analytics so you can form some hypothesis, and then running more specific experiments.

Other important consideration when designing your experiment is to think through what cohorts of users do you want to experiment on. Do you want to experiment solely on new users who have never seen your pricing page before, or you want to experiment with just the free user base.

There are all important considerations in deciding who to experiment on and setting your experiments up for success.