A/B testing on Shopify used to be a bad idea. The general-purpose tools swapped content with JavaScript after the page had already rendered, which produced the flicker everyone remembers, slowed the page, and broke on the templates that mattered. That changed when Shopify let stores serve alternate themes and templates: a test variant is now a real theme, rendered server-side, with no flash of the wrong page.
Three apps use that properly. This is what each one is for, what you can and cannot test, how much traffic you need before the result means anything, and the tests we run first on a Shopify store.
Summary
- Intelligems — the only one built for price, shipping and discount testing, with margin-aware reporting.
- Shoplift — theme and template testing with the cleanest setup for CRO programmes.
- Visually — testing plus personalisation and a visual editor, for teams without a developer.
What you can actually test on Shopify
| Test | How | Needs |
|---|---|---|
| Whole theme (redesign vs current) | Alternate published theme | Any of the three apps |
| Product or collection template | Alternate template assigned to the variant | Any of the three |
| Section content, copy, images | Duplicate section in the variant theme | Any; Visually has a visual editor |
| Price, shipping threshold, discount | Price test with margin reporting | Intelligems |
| Checkout (Plus) | Checkout extensibility blocks per variant | Intelligems or custom, Plus only |
| Email or SMS flow | Your ESP’s own split testing | Klaviyo, Attentive, Postscript |
1. Intelligems
Intelligems is the price-testing app. It splits traffic and tests prices, free-shipping thresholds, discount levels and offers, and reports on profit rather than conversion rate alone — which is the point, because a 10% price cut that lifts conversion 8% is a loss. It also does content and theme tests, so many stores run it as their only testing tool.
Two cautions with price tests: decide in advance how you handle a customer who sees the higher price and finds the lower one (honour it), and keep tests short enough that the same shopper rarely sees both.
2. Shoplift
Shoplift is built specifically for theme, template and section testing on Shopify, with no-flicker rendering, sensible traffic allocation, and reporting that non-analysts can read. If your programme is “we redesigned the product page, prove it is better before we roll it out”, this is the cleanest fit, and it is the one we most often set up inside a CRO and experimentation retainer.
3. Visually
Visually combines A/B testing with personalisation and a visual editor, so a marketer can build a variant without touching Liquid. That is its strength and its risk: editor-built variants are easy to create and easy to leave running long after the test ended. Use it where the team is non-technical and someone owns the cleanup.
Quick comparison
| App | Theme / template tests | Price tests | Personalisation | Editor for non-devs | Best for |
|---|---|---|---|---|---|
| Intelligems | Yes | Yes, with margin reporting | Limited | Partial | Pricing, shipping and offer strategy |
| Shoplift | Yes, its focus | No | No | Partial | Structured CRO programmes |
| Visually | Yes | Limited | Yes | Yes | Marketing teams without a developer |
How much traffic you need
The honest answer is: enough to reach significance within two to four weeks, because a test that runs longer gets contaminated by seasonality, campaigns and stock changes. A practical rule from the stores we run: if a variant will not see roughly 1,000 sessions that reach the cart in a month, you are not testing, you are guessing with extra steps.
Below that threshold, the useful work is not testing at all — it is fixing the things a structured review finds, which is what the 50-point CRO audit checklist and the free CRO audit are for. Test once the obvious defects are gone and the traffic justifies it.
The first five tests worth running
- Product page above the fold. Image treatment, the order of price, variants and trust signals, and whether the add-to-cart is visible without scrolling on a phone.
- Cart drawer versus cart page. One of the highest-leverage structural tests on any store, and the one most often assumed rather than measured.
- Collection template. Card density, the second image on hover, filters visible or hidden on mobile.
- Free-shipping threshold. A price test in disguise, and the one Intelligems is built for; measure profit, not conversion.
- Checkout extensibility blocks (Plus). Trust content, upsells and field order in the checkout itself — see the seven moves that work.
Rules that keep results honest
- One change per test, with the hypothesis written before the test starts.
- Do not stop the moment it looks significant; run to the planned sample size and at least one full week cycle.
- Do not run a promotion in the middle of a test, or exclude those days honestly.
- Measure to revenue per session, not to click-through on the element you changed.
- Delete the losing variant and the app’s leftover code — old variants slow the store down; check with the Store Speed Test.
- Keep a log of every test and its result. The losses teach more than the wins, and they stop the same idea being re-tested every quarter.
That programme — hypothesis, test, log, roll out — is what our conversion rate optimization work is, and the A/B testing guide has twelve specific tests with the reasoning behind each.
FAQ
Can you A/B test on Shopify?
Yes. Shopify can serve alternate themes and templates, and the testing apps use that rather than swapping content with JavaScript after the page loads, which is what caused flicker on older tools. Intelligems, Shoplift and Visually all run theme- and template-level tests with proper randomisation and reporting.
Can I A/B test prices on Shopify?
Intelligems is the app built for it: price, shipping-threshold and discount tests with margin-aware reporting. Test prices carefully — customers talk, and a price test needs a clear policy on honouring the lower price.
How much traffic do I need to A/B test?
Enough to reach significance in two to four weeks. As a rough guide, a store with fewer than about 1,000 checkout-reaching sessions a month per variant will spend longer than the test is worth; below that, fix what a CRO audit finds and test later.
What should I test first on a Shopify store?
The product page above the fold, then the cart, then the collection template — they carry the most traffic and the most decisions. Test one change at a time, with a hypothesis written before you look at the result.
