Wondering if your vendors or agencies are taking advantage of you? Then you need to read about the three most common eCom scams.
This content was originally published in the No Best Practices newsletter on 11.26.2023.
A few weeks ago I tweeted about common eCom scams/red herrings. I got some requests to elaborate on these scams, so that’s what I’m going to do here. BFCM weekend is almost over and 2024 planning conversations will start soon. Don’t fall for one of these “quick wins”.
I had a few folks question if brands really fall for these “gray hat” tactics, and the answer is YES. I spent most of my eCom career at brands that were not founded as DTC. Management rarely had any familiarity with the concepts I talk about in this newsletter. So they fell for this stuff again and again.
I also see first time founders who are new to the eCom and DTC space get fooled by these scams. Read on to protect yourself. Even if you think you can spot a scam from a mile away, it’s good to get a refresher.
eCom Scam #1: “Creative” Attribution and Obfuscation
This scam comes in two flavors:
The first flavor = retention agencies that set up a bunch of email flows for the client, then use really generous Klaviyo attribution to prove “massive added value”. This is even worse if they’re charging you a commission based on the “value added”.
Klaviyo has some attribution models where it tallies up every sale from folks who clicked an email in the last 30 days, or every sale from folks who opened in the past week.
Using these attribution models, sending more email = more attributed dollars. Do those emails actually do anything to improve sales at the business level? Maybe.
If you go from sending one email per week to sending 5x/week + 10 different flows, it will probably generate incremental sales. Just make sure that if you’re paying for incremental sales and not some bogus, overly generous attribution number.
Read more about the right way to measure the impact of retention marketing here.
The second flavor = Martech vendors who “grade their own homework”, using their own black box attribution.
Two vendors come to mind here. One is a much-derided retargeting platform, and the other charges exorbitant fees to place custom javascript widgets on the client’s website.
One thing that both of these vendors (and TBH, most eCom scams) have in common is incredibly aggressive salespeople. I’ve seen sales reps from these brands use social engineering tactics on management to book a call. Not cute.
The attribution scam here is typically comparing users who engaged with the tech vs users who did not engage with the tech. Because of where the tech sits in the marketing funnel, this is never a like for like comparison. Higher intent users are more likely to interact with the tech, making it look like the tech is doing something.
Of course, if you suggest a holdout test to verify this wonky attribution, these vendors will flip their lids.
eCom Scam #2: Outdated Agency Models
I like to say that there are no best practices. But when it comes to digital marketing, there are table stakes tactics you need to employ if you hope to have any chance of success. These tactics can change a lot over time; they’re usually tied to updates in platform technology. If a marketer or agency doesn’t stay up to date, they’ll find themselves offering irrelevant services.
The first of these outdated agency models isn’t always a scam. Sometimes it’s just a team that continues to offer an outdated approach. But… sometimes it’s a scam.
If your Meta ads agency recommends that you run a large number of micro-campaigns targeting different audiences…run!
You cannot succeed with Meta ads running 20 campaigns that are each spending $5/day. Post-iOS14, this approach no longer works. You should be aiming to spend enough to get your campaigns out of “learning mode”, driving at least 50 conversions per week.
I am shocked at the number of Meta accounts I audit where the brand is still using some variation of this approach. When the brand takes their top creative and switches to a more consolidated approach, performance picks up almost immediately.
Another variation is a TOFU/MOFU/BOFU setup where TOFU is a pageview campaign, MOFU is a conversion campaign and BOFU is a retargeting campaign. For most brands, a pageview objective and a retargeting or remarketing objective are a waste of time and resources (this changes when you’re spending >$20k/day consistently).
In addition to outdated channel management practices, there are also outdated agency management practices. An agency should not be charging you 10-15% of your media dollars to manage demand capture activities.
An agency I used to work with at a former employer charged us 12% to oversee branded search, discount-based affiliate and retargeting. They did not understand the difference between demand generation and demand capture. They were handsomely paid project managers. You could find someone on Fiverr to do this work.
Do not let this happen to you. Don’t be impressed by an agency’s client roster of nationally recognized brands. If you are a small and growing brand, a five-star roster is an indication that the agency you’re considering is the wrong fit for you. More on that here.
eCom Scam #3: Stats Abuse
I know statistics is confusing. But you should make an effort to learn the basics, because there are a lot of unsavory characters looking to take advantage of this fact. Or maybe they’re just as confused as everyone else? Either way, the end result is brands investing a lot of time and money into programs that don’t actually make sales go up.
CRO and A/B testing is the domain where stats abuse is most frequent. One common scam is when a CRO agency assures the brand that statistical significance doesn’t matter that much (or fails to mention it at all).
The agency then runs an endless stream of micro tests–blue vs red buttons!–and calls winners regardless of statistical significance. The brand feels like they’re making a ton of progress implementing all these “winners”, but the site’s conversion rate remains unchanged.
If your brand is doing under $50M/year in revenue, you will usually have to run a web A/B test for at least two months to get a stat sig result. So you can run six tests a year. Better make them meaningful (hint: button color is not meaningful).
A slightly more sophisticated scam is when CRO teams or agencies run stat sig tests for four weeks, take the lift, multiply it by 13, and claim that they “drove $1.2 million in value”.
You can’t do that! The lift from most web UX tweaks regresses to the mean within 3-6 months–that “$1.2 million in value” becomes a goose egg. So if you want to claim some annual value number, you need to keep a holdout running and monitor it over time.
Bonus eCom Scam: Most Loyalty Programs
Loyalty software vendors NEVER want to run a holdout test of the program in a closed beta. They always want to lock you into a big, splashy, public launch so you can burn your boats and continue paying their monthly fees in perpetuity.
This is the essence of most loyalty program case studies:
- We implemented a points program from (insert SaaS vendor here).
- Customers who are enrolled in the program spend 3x more than customers who are not.
- Insert quote from director or VP of eCom/marketing about Starbucks and Sephora.
This makes smoke come out of my ears. Especially when these case studies convince smaller brands that a loyalty program is the solution to their growth issues (it never is).
The reason that loyalty program customers appear to spend more? Because the customers who make the effort to enroll in your program and jump through its hoops are already predisposed to spend more.
Saying that “Loyalty program members spend more” is like saying “Millionaires spend more than people living below the poverty line”. It may be true, but it’s just an observation–your marketing didn’t make that happen.
What you really need to compare are Loyalty members and non-members from a cohort of customers with similar levels of engagement. The best way to do this is a holdout test, performed before you launch the program to the general public.
The second best way to do this is to compare L12M customers in your Loyalty program vs those who are not in the program, but split the comparison by lifetime order count. Ex. 3x buyers who purchased L12M who opted in to Loyalty vs those who did not.
The loyalty SaaS crowd hates when I tweet about this, but I’m still waiting on a stat sig holdout test that validates any of their software. If one of these vendors approaches you, do yourself a favor and request a private holdout test to validate the software before you sign any contracts.
If they cooperate, email me and I’ll eat my hat/give them a positive mention.
Fun little caveat to all this advice: if you work for a large organization that is not digitally sophisticated, you can use all these techniques to your own advantage, scoring easy “wins” to get yourself promoted. Do with that info what you will.