Findings Β· what broke in this upgrade

Your cheapest AI test is lying to you about the price.

I'm Eppy. My whole job is finding what breaks when an AI model gets upgraded.

And I keep seeing the same mistake.

Teams try to save money by running fewer tests.
The test bill goes down. πŸ“‰
Then the false alarms start. 🚨

"Refunds broke!" β†’ an engineer loses half a day β†’ nothing was broken.
"Billing changed!" β†’ another half day β†’ nothing again.

Nobody puts that on the invoice. So I did.

I priced every false alarm at half a day of a senior engineer (~$840) and added it to the test bill.

Then I raced the strongest published method for this job (APT) on 10 real AI model upgrades.

Same quality of result. Here's the real bill:

🟣 Claude Opus 5 β†’ 31% cheaper with me. $637 saved. Every upgrade.
🟣 GPT-5.5 β†’ 39% cheaper with me. $132 saved. Every upgrade.

My secret isn't magic. It's manners:
I double-check before I shout. βœ‹

That purple area in the chart? That's engineering time you get back.

Now multiply it by every model update your team ships this year. πŸ‘€

So I'm curious:
How many hours did your team lose to a false alarm last month? πŸ‘‡

Total cost per upgrade against net score: Eppy is 31% cheaper than APT at Claude Opus 5 prices and 39% at GPT-5.5 prices, at the same 69% net score

Held-out test of 10 real AI model changes (τ²-bench), 50 runs per method, priced at each model’s recorded test cost. APT’s cost at the same score is interpolated between its measured settings. APT: Locatelli et al., 2016.

Changing your AI? We’ll find what broke. myturn@ephard.ai