We’re thrilled to introduce A/B Experiments for Cancellation Benefits Pages. Brands can now test different retention content strategies by playing with content format, messaging, visuals and more to uncover the most saving benefits pages.

Key highlights

• Test different content formats: Experiment with video vs. text, GIF vs. video, or messaging in founder's video to see what resonates most with your subscribers.
• Simple setup: Create your benefits page in a few simple steps by configuring audience split, winning criteria and importantly content for control and variant group.
• Side-by-side preview: Easily compare control (A) and variant (B) content design, allowing for quick feedback and visual insights needed for iterations.

How to create an experiment?

• Audience split: Define the percentage audience split between control (A) and variant (B). Subscribers will be randomly assigned to A or B design when they click to cancel.
  
  

• Winning criteria: Set a minimum delta in save rates to declare a winning variant. Example:

  • Case 1: Control (A) - 10% saves vs. Variant (B) - 12% saves = B wins (20% more saves)
  • Case 2: Control (A) - 10% saves vs. Variant (B) - 11% saves = No winner (less than 20% improvement)
    
    

• Experiment completion: Define the experiment duration based on

  • Number of days
  • Number of cancellation attempts

Performance tracking:

• A/B experiment tracking is available in Loop Admin under Tools and Apps, where you can track:

  • Save rate
  • Attempts & saves
  • Day-on-day trends
• Export results to analyze which variation was served subscriber IDs, content variants, and outcomes.

What happens when experiment ends?

• Once experiment ends, the control design will automatically start getting displayed for all the subscribers.
• However, up to 7 days post-experiment end, both content versions will be visible and brands can decide which variant to continue with.

Once experiment ends, the control design will automatically start getting displayed for all the subscribers. However, up to 7 days post-experiment end, both content versions will be visible and brands can decide which variant to continue with. 

Start experimenting now to find the content that drives the most savings and conversions for your subscribers!