📖 Boost's A/B Testing for Merchandising allows you to experiment with your merchandising rules, helping you determine which strategy maximizes your store's conversions and profitability.

⚠️ This article only works for Boost AI Search & Discovery Turbo and Professional plan. To find out which version of our app your store's current theme is installed with, please follow this article.

Overview

With A/B Testing, you can create a variant of an existing merchandising campaign (the control) and run them in parallel. The app will automatically split customer traffic between the two versions and track their performance based on your setup.

This feature supports both Search and Collection merchandising types and allows you to measure results using key goals such as conversion rate or revenue.

Key Concepts

Element	Description
Control version	The original merchandising campaign you want to test.
Variant version	The modified campaign used to compare with the control.
Test duration	The start and end dates for the A/B campaign. Must fall within the control campaign’s active period.
Traffic split	How customer traffic is divided between the control and variant (default is 50/50).
Test Goal	The success metric of your test. Currently, Conversion Rate and Revenue are supported.

Before You Start

Please note the following conditions:

You must already have an existing merchandising campaign.
Only one A/B test can run per merchandising campaign at a time.
You can only test specific search terms or collections. Global rules (e.g. "All search terms") are not supported.
If your store has global rules (e.g., apply to all collections/search terms), they will apply to both control and variant.
An A/B test must be created within the date range of the associated control campaign.
Traffic is assigned based on user IP to ensure balanced comparison.
The system automatically activates or ends campaigns at 00:00 UTC daily.

How to Create an A/B Testing Campaign

Access A/B Testing
- From your Boost admin panel, click on the A/B Testing on the top-right menu (If this is your first time, a short onboarding popup will appear).
Click “Add New Campaign”
Set Campaign Details
- A/B testing campaign name: Add a name for internal reference.
- Targeting: Select the existing Search or Collection merchandising campaign.
- Test Duration: Choose the date range (must be within the campaign’s active period).
- Test Goal: Select a goal—currently defaulted to Conversion Rate.
Configure Variant
- Your existing merchandising campaign will serve as the Control version
- Set new merchandising rules for the Variant 2.
- These settings work just like regular merchandising rules.
- A variant must be different from the control version.
Adjust Traffic Split
- Default is 50/50, but you can customize (e.g., 60/40).
  - The actual distribution may not be exactly equal. Variations of approximately ±10% can occur due to real-time traffic and user behavior. For example, the 50/50 split might result in a 43/57 or 47/53 distribution. This is expected behavior and does not affect the validity of your test results.
- Each version must receive at least 10% of the traffic.
Review and Launch
- Double-check the details.
- If your campaign is scheduled for the current time, it will go live immediately under the Live tab.

📝 If your A/B test is scheduled for the future, it will be saved under the Draft tab until the start date.

Managing A/B Testing Campaigns

Tab	Description	Action
Live	Lists all ongoing A/B tests. You can rename or extend campaigns here.	Edit End Delete See analytics
Draft	Shows upcoming A/B test campaigns. You can edit before they go live.	Launch Edit Delete
Ended	Displays completed campaigns. Review analytics and identify the winner.	Delete See analytics

⚠️ If you uninstall Boost or downgrade from the Professional plan, all active A/B tests will be paused and locked.

To manage your campaign individually, click the "⋯" button under the Actions column.

Action	Description
Launch	Start the scheduled campaign immediately.
Edit	Live: You can only change the campaign's name and duration (only within merchandising rule time). Draft: You can change all the campaign's settings.
End	End the ongoing campaign immediately. The campaign is still accessible internally for further tweaks if needed.
Delete	Delete the campaign (while also ending it). The campaign will be removed completely and no longer accessible anywhere.
See analytics	Review the campaign's analytics and view the winning variant.

Feel free to reach out to our dedicated support team via chat if you have any questions or require additional assistance.

Ranking a group of products in search results

Merchandising rules for Collection Page

Merchandising Rules for Commerce Search

Basic flow to set up a Boost Recommendation widget