Your First A/B Test: Discount Depth vs. Product Count

You have been running campaigns for a few weeks. Some convert well, others do not, and you are not sure why. Was the discount too small? Were there too many products? Too few? Without a structured test, you are stuck guessing.

A/B testing eliminates the guesswork. It splits your customer group into segments, gives each segment a different strategy variant, and lets you compare the results side by side. The split is randomized, so differences in outcomes are attributable to the strategy — not to which customers happened to be in which group.

The test: discount depth vs. product count

For your first A/B test, focus on the single most impactful variable: the tradeoff between how much you discount and how many products you show. Set up two variants:

Strategy A: Bigger discounts, fewer products

• Discount range: 25-40%
• Products per catalog: 4
• The pitch: "Here are a few items we think you'll love, at prices you won't see again."

Strategy B: Smaller discounts, more products

• Discount range: 10-20%
• Products per catalog: 8
• The pitch: "Browse your personal collection — something for every mood."

Keep everything else identical: same customer group, same campaign duration, same email template. The only difference should be the strategy variant. If you change multiple variables at once, you will not know which one caused the difference.

One variable at a time. It is the oldest rule in testing because it is still the most important one. Change the discount and the product count together, and you learn nothing actionable.

What to measure

Three metrics tell you almost everything you need to know:

1. Conversion rate (redemption rate)

Orders divided by catalogs sent. This is your primary success metric. If Strategy A converts at 12% and Strategy B converts at 8%, bigger discounts with fewer products are winning for this audience.

2. Revenue per catalog

Total revenue from the variant divided by catalogs sent. This is your ROI metric. Strategy B might have a lower conversion rate but generate more revenue per catalog if customers who do convert buy higher-priced items or add more to their cart.

3. Average order value (AOV)

Total revenue divided by number of orders. Compare this to your store-wide AOV. Strategy A (deeper discounts) might produce a lower AOV per order but higher conversion. Strategy B (more products) might produce larger carts because customers have more items to browse.

How many customers you need

A/B test results are only meaningful with enough data. The minimum you should aim for:

• 100 customers per variant — This gives you enough data to see a real pattern. With 50/50 split, that means a customer group of at least 200.
• Full campaign duration — Do not check results halfway through and call it. Some customers convert on day one, others on the last day (especially with reminder emails). Let the campaign run its full course.
• At least 10 conversions per variant — If a variant has only 3 orders, the results are not statistically meaningful no matter how large the group. If neither variant hits 10 conversions, run the test again with a larger group or a more compelling offer.

Common first-test results

While every store is different, here are patterns that show up frequently:

• Fashion and apparel tend to favor more products (Strategy B). Customers like browsing and the variety drives discovery.
• Electronics and home goods tend to favor deeper discounts (Strategy A). Customers already know what they want; the discount is the deciding factor.
• Beauty and personal care are often a toss-up. Both approaches can work, which makes A/B testing especially valuable in this category.

What to test next

Once you have a winner from your first test, resist the urge to test five things at once. Follow this sequence:

1. Discount depth — You just did this. Lock in the winner.
2. Product count — Test 4 vs. 6 vs. 8 with the winning discount range.
3. Collection scope — Test full catalog vs. specific collection strategies with the winning depth and count.
4. Customer segment — Run the winning strategy against a different customer group to see if it generalizes.

Each test builds on the previous one. After three or four rounds, you have a strategy that is genuinely optimized for your store — not based on best practices from a blog post, but on actual data from your customers. Track it all with your campaign results dashboard.

When not to A/B test

If your customer group has fewer than 100 people, skip A/B testing and run a single strategy instead. You will learn more from comparing two separate campaigns (run a week apart) than from splitting a tiny group. A/B testing is powerful, but it needs volume to be reliable.