Starting an Experiment

You run experiments in Superwall by adding multiple paywalls to an audience. To start an experiment:

1. Select an audience.
2. Click the Paywalls tab.
3. Add two or more paywalls.

Here's a .gif, beginning to end, setting up an experiment:

It's as simple as that to start a paywall experiment.

Setting presentation percentages

You must set a presentation percentage between your paywalls within the experiment. This determines how often they'll show based off of the percentage set for each one.

To set a percentage, click the pencil icon above any of the paywalls attached to the audience:

Then, assign percentages from 0%-100% for each of them. In total, your percentages should equal 100 (i.e. paywall A shows 10%, paywall B shows 30%, and paywall c shows 60% of the time) unless you're purposely creating a holdout. When you're done, click the Checkmark icon below any paywall:

Resetting assignments

If you change your experiment, or simply want to change the presentation percentages between your paywalls, you might want to reset your assignments. Remember that when an audience matches a user, it's sticky — and the same is true of when someone is matched to a paywall within an experiment.

So, if you want to make sure everyone is matched again to a paywall based off new percentages, click the refresh button below a paywall when editing percentages (next to where it says "X assigned"):

Creating holdouts

A holdout is a share of an audience that sees no paywall. It acts as a control: compare it with the users who saw a paywall to learn whether showing one there pays off. Give the paywalls less than 100% of the audience, and the rest is held out.

For how to set the split, what your app sees for held-out users, and how to read the results, see Holdout Groups.

Removing variants

During an experiment, you may find that one or more paywalls are performing significantly worse than the others. In that case, you would probably consider removing it. You can simply remove the paywall, or set its presentation percent to 0%, and your experiment will continue. No metrics will be affected or reset. Resetting assignments will reset metrics, removing paywalls will not.