Every ad is five decisions, and a winner proves the combination
Curtis’s system starts with a simple idea: every ad is really five decisions made at once. There’s the product you’re selling, the angle that makes someone care, the creator saying it, the offer that gives them a reason to buy now, and the format, meaning how it’s shot and cut.
Let’s say a skincare ad takes off. A creator films her morning routine, mentions a 20% off code, and the ad beats your target CPA for weeks. It’s tempting to call the creator your winner and book her for ten more videos. But was it her? The morning-routine angle? The discount? Or the fact that she filmed it in one take on her phone?
As Curtis put it, when an ad wins, it’s all five pieces coming together. That doesn’t mean all five were winners, and it doesn’t mean any single one was. A winner proves the combination, not any single piece. If you don’t know which of the five changed between two ads, you can’t say what made one of them work, and that’s where a lot of testing goes wrong.
Every new variable cuts your hit rate
There’s a second reason to limit what changes: every unproven piece lowers the chance an ad works.
Curtis illustrates it with simple odds: if each new piece has a 1-in-3 chance, two new pieces drop you to 1 in 9, and three to 1 in 27. In practice, his best case is about 30%, when a really strong creator is only tweaking what they say about the product.
Every ad costs money to produce and more to test, and testing is often the bigger cost. So keep most tests to one or two new variables. Big swings are fine, just not every time.
Change one or two things, big enough to see
Here’s what that looks like, with most of the five pieces proven and one or two changed on purpose.
Curtis’s own example of a creator test looks a lot like the skincare ad from earlier, with one big difference: only one piece is new. The product, the morning-routine angle, the 20% off offer and the UGC format have all worked before. The creator is the only thing being tested, so whether the ad wins or flops, you know why. And whatever you change, make the change big enough to tell apart at a glance, so the result doesn’t get lost in the noise.
This is exactly how his team works with Billo. When it needs new creators, it sends four proven combinations through Billo and watches who delivers. Once a creator has proven themselves, the team flips it around: it keeps the creator and gives them a new product, offer or angle to test. Because only one piece changes each round, the learnings add up from one test to the next.
Curtis didn’t always work this way, though. He learned it the hard way, on his very first Billo test. His team used it to try a brand-new product with new creators, the ads didn’t work, and he wrote Billo off. Only later did the team realize the product itself never worked, with any creator, anywhere. When they came back to Billo with their core product, the creators performed really well. A new channel, new creators and a new product all at once leaves you with nothing to learn from.
“One of the flaws I see that I’ve made, and that I’ve seen other people make, is testing too many variables… I would say do the opposite.”
Curtis Howland, VP of Marketing, Misfit Marketing
The creator’s handle is one more variable
Whose handle an ad runs from counts as a variable too, and it’s one of the strongest Curtis has found.
“A creator handle multiplies a good ad. It won’t save a bad one.”
Curtis Howland, VP of Marketing, Misfit Marketing
When his team runs the same content from a creator’s handle as a partnership ad, it performs better about 80% of the time. That holds even for creators with no following who only make ads, including creators who only create on Billo. With real influencers, it’s closer to 100%.
Here’s how to run creator content as partnership ads.
Read the answer on spend, not CPA
Once a test is live, the next question is how to read it. Back to the two ads from the start.
Newer brands, and AI tools, often pick Ad A for its lower CPA. But Ad B is the ad Meta chose to spend on, and its CPA is still under target. That’s the one to learn from and build on.
His bar for a hit is $3,000 to $4,000 in spend at a CAC that works. Below that, a low CPA is often just a small sample. So read spend first, then CAC: judge creative by what Meta is willing to spend on it.
“What we care about is what scales sustainably at high performance. As long as it’s under your target CAC and it’s scaling, that’s a good sign.”
Curtis Howland, VP of Marketing, Misfit Marketing
Every result should lead to a decision
Once you can read a result, it should lead to a decision. Two questions tell you which one: did the ad earn spend, and did the CAC hold while it did?
- Iterate. Give winners new hooks, creators and edits inside the same concept. Here’s how to turn one winning angle into many variations.
- Scale. Put more budget behind what holds up as spend goes up.
- Cut. Pull budget from what can’t earn spend. Expect about 70% of launches to end up here, and that’s normal.
More spend needs more concepts
Cutting most launches means you need a steady supply of new ones, and the more you spend, the more you need. Curtis’s rule of thumb: one new ad for every $3,000 of spend. That’s about 10 new ads a month at $30K, and about 100 at $300K. Anish Dalal of Sapphire Studios, also on the panel, works with a similar $3,000-5,000 range.
Production is the constraint: the more an ad costs to make, the more spend it has to earn.
“High quality and volume, if you can make it.”
Curtis Howland, VP of Marketing, Misfit Marketing
To check your pace, divide total spend by the ads you launched in the last 30 days.
It’s a rough guide that shifts with spend level and category, but it shows quickly whether you’re over- or under-testing.
The same math works for comparing creative sources. Cost per asset is what you paid; spend per ad is what Meta wanted. Let’s say one agency launched 10 ads that earned $100K in spend, and another launched 100 that earned $110K. Curtis would take the first, because each of its ads earns more. Run the same check for in-house work, agencies and creators.
That only works if your ad names roll up. Curtis uses the same order on every ad, such as C14-Routine_JessM_UGC_v3: concept, creator, format and iteration, plus who produced it.
If you’re starting from zero
All of this assumes you already have winners to build on. With no learnings or data yet, Curtis’s advice for a first round of tests is:
- Start with video. It carries more information. Statics tend to work better once you’ve scaled.
- Start with why you started the brand: your selling points, your customers and what you care about.
- Give each test enough budget to learn. Meta’s guideline is 50 conversions per ad set per week, so work your budget back from your CPA. At a $40 CPA, that’s about $2,000 a week per ad set. Anish suggested at least $5,000-10,000 a month for good signal, and Reyna Baker of Meta added that a small budget with only a few ads won’t get out of the learning phase.
How to apply this to your next creative test
- Change one or two of the five variables per test, big enough to see.
- Put strong creator content on the creator’s handle.
- Read spend first, then CAC. A hit is $3,000-4,000 in spend at a CAC that works.
- Iterate, scale or cut every ad, and expect to cut most launches.
- Check your spend per ad monthly, and name ads so results roll up.
For the mechanics of setting up tests, see our ad testing guide. If you’re ready to test new creators on a concept that already works, you can find creators on Billo.
“Every ad should teach you something.”
Curtis Howland, VP of Marketing, Misfit Marketing
