Introduction: Why Smart Price Testing Matters and
How to Find the Optimal Price with Minimum Experiment Spend
Smart
e-commerce price optimization finds the sweet spot for profit, fast and cheaply. In this article we’ll explore an innovative approach: sequential price testing with no price discrimination (i.e., every customer sees the same price at the same time), combined with a clear fairness policy to explain price changes, and a powerful underlying model from the world of machine-learning: the
multi-armed bandit. We’ll compare this to traditional A/B testing, illustrate the benefits for e-commerce, include a real-world case study, and provide a practical framework you can implement in your own business.
By the end of this page, you’ll have both the conceptual understanding and the practical roadmap to apply
dynamic price testing in your e-commerce business — with lower spend, less risk, and faster convergence to the optimal price.
1. Why Price Testing Matters in Modern E-commerce
The Importance of Price Testing for Online Retail
Effective pricing is the balance between maximizing willingness to pay (WTP) and maintaining perceived value (Vp). If you miss the mark, you either leave money on the table or sacrifice sales volume. You either sell a lot of units, earning less profit per unit than you could or you are losing potential customers who would have bought the product if the price had been slightly lower.
The Challenge of Guessing the “Right” Price
Modern strategy for price testing is about finding that specific sweet spot—whether it’s a higher price point or an optimal discount—to drive consumer behavior. Many merchants use historical data, competitor benchmarking or cost-plus pricing. But these approaches don’t adapt to market shifts, and they often ignore demand elasticity, inventory sell-through, promotional lift, and competitive reaction.
From Surveys and Spreadsheets to Data-Driven Experimentation
Rather than only asking “what price would customers pay?”, modern approaches ask “what price
should we set now given the context?” A well-designed price test reveals this by measuring real behaviour under real conditions at scale. This means testing prices of multiple products simultaneously. Segmenting on collections, categories, vendors or tags, merchants can find the winning prices in bulk.
2. The Pitfalls of Traditional A/B Price Testing
What Is A/B Price Testing in E-commerce?
At its simplest, A/B price testing means you choose two (or more) price points, randomly split traffic between them, and compare key metrics (conversion, revenue, profit). Over time you pick the winner and roll it out.
Why A/B Testing Can Be Costly and Slow?
-
- – High experimentation spend: While you run the price test, one variant may be sub-optimal, losing revenue or profit.
-
- – Fixed traffic allocation: You continue allocating traffic to both variants even after one looks clearly worse.
-
- – Long duration: To achieve statistical significance you may need many conversions, especially if price effect is subtle or segmentation is needed.
-
- – While the vast majority of modern A/B price testing tools are designed to run simultaneously—splitting incoming traffic in real-time to ensure external factors like seasonality affect both groups equally—this methodology introduces an unfair flavor of price discrimination, as two identical customers shopping at the exact same moment may be offered different prices simply based on the randomized variant they were assigned.
3. Sequential Price Testing via Multi-armed Bandit Framework: A New Standard for Fair and Low-Cost Experimentation
What Is Sequential Price Testing in the First place?
In sequential price testing, you deploy a price for a defined time-window (e.g., one hour, one day) to
all users so every user in that window sees the same price and then moves to the next candidate price.
Why Sequential Multi-armed Bandit Approach Reduces Experimentation Cost
-
- – You test fewer price candidates and switch when one clearly under-performs.
-
- – Because you allocate more traffic to a winning price automatically
-
- – You don’t need to monitor the price test outcome closely—the model naturally converges to the optimal price over time.
-
- – You ensure fairness: no customer group receives a worse price for longer than needed.
No Price Discrimination: Identical Price for All Users at the Same Moment
A key principle: at any given moment all users see the same price. This avoids ethical or reputational risk of “two customers paying different prices” based on hidden attributes. It also simplifies segmentation and statistical analysis because you don’t need to factor in individual price exposure differences. In some geographies price discrimination is forbidden by law and the merchant risks violating it, if the prices are simultaneously changed.
If you are a Shopify merchant, you can install
DynamicPricing AI Optimization and get better understanding of how this innovative price testing technology works.
4. Introducing the Fairness Policy: Transparent Price Testing
When running price testing experiments powered by a multi-armed bandit model, prices may move up or down over time as the system learns which price performs best. Since the algorithm relies only on PDP visits and orders — and does not have access to personal data or user-level attributes — your fairness policy should explain clearly why these price changes occur and how you guarantee that every customer is treated equally.
Why Prices May Decrease
The price testing models may recommend a lower price when the aggregated behaviour suggests that:
- – Conversion rate meaningfully improves at a lower price.
- – A lower price generates higher profit per visit or revenue per visit.
- – The system needs to “explore” lower price variants to confirm whether performance is superior.
- – Market conditions shift (e.g., increased PDP traffic, external demand factors) and a lower price becomes more favourable.
All decisions are based on simple signals: how many people visit the product page, and how many complete a purchase.
Why Prices May Increase
Higher prices may be selected when the price testing (the multi-armed bandit) algorithm observes that:
- – PDP engagement remains strong despite higher prices.
- – The higher price produces better margins or higher profit per visit even with slightly lower conversion.
- – More exploration is needed at upper price ranges to understand potential upside.
- – Customer purchase behaviour indicates willingness to pay more for the product.
