Understanding Experiment Results

To view the results of any paywall experiment that’s running, click the Results tab in the campaign details view:

There are three main sections: Paywalls, Placements, and Graphs (defaults to Proceeds Per User). Each section has a toggle at the top right to change associated metrics.

​

Paywalls

Here, you’ll see each paywall being used (or that was used) in an experiment. Superwall will show you metrics such as proceeds, users and much more. There are several metric to explore, and you can hover over any of them to get more details about what each metric represents:

You can also filter results per paywall. Click the checkbox next to one to have the results page only show data for that specific paywall:

​

Placements

Here, you can get a detailed breakdown of each placement associated with the campaign. This helps you form a clear picture of what features or actions are leading to conversions.

​

Graphs

Finally, the last section has several graphs to explore campaign performance. It defaults to Proceeds Per User.

​

Setting up revenue tracking

Before any metrics based on revenue will display, you need to setup revenue tracking. To setup revenue tracking:

1. Click on Settings in the dashboard.
2. Click on Revenue Tracking.
3. Use the guides to follow any of the revenue tracking methods. For more details, check out our docs.

If you don’t have revenue tracking setup, you will see a banner on your dashboard:

​

A note on conversions, trial starts, and subscription starts

Each experiment will notably report conversions, trials starts and subscription starts. In some cases, it may seem like these numbers don’t match up quite how you’d expect. That could be due to a few different reasons:

1. Reporting methods: Conversions are an SDK reported event, while trial and subscription starts are server reported events. Sometimes, the server events might be a little behind on their reporting — whereas SDK events are usually instantaneous.
2. Understanding Resubscriptions and Cancellations: When someone resubscribes or restarts a paused subscription through a paywall, it won’t be considered a new trial or a new subscription start. However, it will be counted as a conversion. As such, any revenue generated will be linked to that paywall. If they later decide to cancel the subscription, the cancellation will also be linked to the same paywall.
3. Attribution: And finally, attribution can sometimes be a complicated metric to track. If something doesn’t look right on your end, please feel free to reach out to us and we’ll always export your data so you can exactly where our numbers are coming from.

​

Confidence intervals

Use confidence intervals to gauge how each paywall is performing against the other ones in your experiments. Hover over a specific metric to view the confience interval (i.e. Conversion Rate, Proceeds Per User, etc.):

Keep in mind that these intervals represent the percentage of users converted, it doesn’t take into account revenue. Put differently, paywall A could have a higher conversion rate, but with a much cheaper offering than paywall B. Paywall B could still be making more money, but at a lower conversion rate with the higher-priced product.

For more on confidence intervals, check out our in-depth blog post.

​

Identifiers and cohorting with 3rd party analytics

If you scroll to the end of the experiment results table, you’ll find some useful identifiers which you can use to interface with third-party tools you may be using:

1. Experiment id: The identifier of the experiment that the paywall is a part of.
2. Variant id: The identifier representing the variant the paywall represented in the experiment.
3. Paywall id: The identifier for the paywall in the experiment, which associates back to the variant.

To learn more about interfacing with 3rd party analytics, check out this doc.