To enable A/B Testing during notification creation simply click the switch next to "Testing" in the Message section header:
Once testing is enabled additional options will be shown on the page. First, a new bar will appear that contains each variant that will be tested for this notification. Up to four variants can be created by pressing the plus sign (+).
Once you've created each variant you want to test, scroll down to the "Test Configuration" section of the page. This section contains configuration options that allow you to name your test and determine how your test will be split and deployed to your audience.
Naming your test is important for future identification of notifications that comprise a test as well as identifying performing within the Insights section of the platform.
Distribution types allow you to determine how and when your test variants are deployed to your audience.
Split Delivery allows you to choose a percentage of your audience that each variant should receive. This distribution type allows you to test new variants on small percentages of your audience while ensuring clickthrough rates stay high with what you already know works.
Sample Delivery delivers a sampling percentage of your audience to each variant before deploying the best performing notification to the rest of your audience.
Use the sliders to determine what percentage of your audience each variant and the winner will receive. Each variant must receive at least 5% of the audience's delivery.
When running a Sample Delivery test, you have the option to activate a performance constraint. If none of your deployed variants in the A/B Test reach the CTR value placed in the performance constraint, then no Notification will be deployed to the remaining audience.
When A/B Testing is enabled additional options appear within the
Scheduling section of the notification creation page depending on the distribution type chosen.
When Sample Distribution is chosen a
Sample Winner section appears that allows you determine when the variant with the best performance is deployed to the rest of your audience. There are two ways to configure this timing:
Time Elapsed After Send: Specify the number of hours that have elapsed after the initial variants were sent.
Specific Time: Schedule the winning notification to be delivered at a specific date and time in the future.
When Split Delivery is chosen an additional switch will appear that when enabled allows you to choose different delivery times for each variant.