What the
work looks
like.

The CFO's stated problem was "fraud is too high." That wasn't actually the problem. Fraud losses were within acceptable industry range — 14 basis points on transaction volume, against a 12bps target. The real problem surfaced in the customer support tickets: an outsized share of customer churn was tied to legitimate transactions being blocked. The fraud system was succeeding at fraud and quietly destroying the top of the funnel.

The Diagnose phase reframed the success metric: not "reduce fraud losses" but "reduce false-positives while holding true-positive recall constant." Once that was stated, the engineering team — who had felt blamed by the original framing — became active collaborators.

Two architectural decisions did most of the work. First, we replaced the binary block/allow decision with a four-tier confidence score — high-confidence fraud blocked, high-confidence-legitimate passed, and two middle tiers routed to either step-up authentication or human review. The middle tiers carried 22% of transactions previously blocked and converted 78% of them.

Second, we built a shadow scoring pipeline that ran the new model alongside the old one for six weeks before takeover. This gave the risk committee data to override on — they could see exactly which transactions the new model would have decided differently, and what the outcomes were.

The Meta3Agents infrastructure compressed the build phase considerably. The shadow-scoring orchestration, the feature store integration, and the human-review queue were configurations of existing capabilities rather than ground-up builds.

The headline number understates the impact. The downstream effect of unblocking 3.4% of transactions was a measurable lift in 30-day retention for new customers — about $4.2M in annualised retained revenue, against an engagement fee an order of magnitude below that.

The forecasting team had a respectable model — gradient boosted, well-featured, ~6% MAPE on stable categories. Then a category-wide regulatory change shifted demand patterns by 40% in six weeks. The model couldn't see the change coming and overstocked one direction while stocking out the other. $11M of inventory imbalance in a quarter.

The leadership team's initial framing was "the model isn't smart enough — let's get a better one." The Diagnose phase pushed back. No forecasting model can predict a regulatory shift it hasn't seen before. The right question was: how does the system know when its own predictions are about to fail?

We left the forecasting model untouched. We built a regime-detection layer sitting above it that monitored for distribution shift across 14 input signals — pricing elasticity, search volume, returns velocity, competitor stock-in indicators, regulatory news embeddings, and seven others. When the regime layer detected a structural break, it would degrade the model's prediction weight and route SKU-level decisions to a conservative envelope.

The second piece was the fallback envelope itself: not a worse model, but an explicit, simple rule-based system that the inventory team understood and trusted. When the AI couldn't see clearly, the system handed back to the humans on rails the humans had helped design.

This is also the case where the co-invest mechanism activated. The regime-detection layer was clearly productisable — it had value to other retailers facing similar volatility. We spun it out as a separate venture, with the original client as the anchor customer and a small equity stake retained.