Samcart vs Shopify - Which Converts Best?
Hypothesis
Is there a difference in conversion when using Shopify or SamCart? I've been using Shopify as my main checkout for selling books. I've seen a lot of hype around SamCart and wanted to see if it would convert better.
This a split test I ran in 2024.
Method
Problems
Problems to account for:
- Shopify has multiple sources of traffic. I need to be able to filter for traffic from my ads.
- Need to split traffic between Shopify checkout and SamCart
- Need to choose good metrics to compare the two checkouts
Test Setup
Here's how I setup the test:
- Cloned my sales page
- Page 1 has links to Shopify with UTM code (important - see appendix for UTM)
- Page 2 has links to SamCart
Additional problems:
- When selling the physical book, I needed the order to be sent from SamCart to Shopify, so that my fulfilment company would get the order and ship it.
- I used Pabbly Connect for this, it's basically a cheap Zapier.
The UTM code on Page 1 allows me to filter Shopify reports to find the sales/traffic coming from the ads.
I ran the test over three days (my SamCart trial was running out and I had only just figured out how to hook up everything and test properly).
The three metrics I chose to measure were:
- Conversion
- Average Order Value
- Dollars per traffic
How do we decide on the winner?
While conversion and AOV are interesting and tell us something useful, what we ultimately want to know is, for a given number of visitors, which checkout makes the most money?
And the best way to measure that is Dollars per Checkout. So for every person that lands on the checkout page, how much do we earn?
This accounts for the interplay between AOV and conversion.
Calculating Dollars per Checkout
Conversion x AOV = $ per checkout
So if the checkout converts at 15%, and the AOV is $20:
Conversion x AOV
15% x $20 = $3
The Dollars per Checkout is $3.
Setting up Samcart
SamCart is significantly more expensive. My shipping and order tracking is done through Shopify, so I needed to integrate the two services using Pabbly Connect.
SamCart also required additional pixel integration to get the Facebook Pixel working.
Results
The results are pretty conclusive:
Test Limitations
It's worth noting that this test wasn't a strict split test that changed one variable - the cart.
The SamCart checkout had two bump offers, the Shopify checkout had none. It could be that conversion was affected by the bump offer (or amount of bump offers). A good further test (if you were willing to spend the dollars) would be to have four variants of the SamCart checkout:
- No bump offer
- Bump offer 1
- Bump offer 2
- Bump offer 1 and 2
It's also worth noting that the SamCart checkout had two large images with the bump offers. I suspect these would have decreased conversion, as they would have interrupted the user experience. However, prior to the test I'd had a call with a SamCart expert, from SamCart themselves, who recommended I do this.
Could SamCart in fact be better?
If you had an upsell offer that was killing it, this could increase the AOV value enough to generate more Dollars per Checkout.
We can calculate how much this would be, by creating a Dollar per Session greater than Shopify, let's say $7, and using the SamCart conversion that we know to be 18.1%
Conversion x AOV = $perCheckout
0.181 x AOV = $7
$7 / 0.181 = AOV
AOV = $38.67
The AOV for Shopify is the average of selling a PDF and a paperback book as a front-end offer, so the difference between these two AOVs will give us the increase in AOV we need from an additional offer in SamCart:
SamCart ideal AOV - Shopify Actual AOV = Offer AOV
$38.67 - $11.10 = $37.57
Let's say we have a good upsell that converts 15% of the time.
$37.57/0.15 = $250.47
So in order for SamCart to be slightly better than Shopify, we need a $250 upsell that is converting very highly.
If we want SamCart to be significantly better, we need a follow up program (in which case... why not follow up with Shopify sales? Email is email), or we need an upsell with even better conversion or of even greater value.
Conclusion
Shopify has a better conversion, so I am sticking with it.