Getting past the 78% wall: the accuracy curve nobody shows you

Almost every AI ops project stalls in the same place, at roughly the same number.

You wire up a model, point it at a real workflow, and it's genuinely useful fast. It gets to somewhere in the high 70s and then it stops. Not because it broke — because it flattened. The next month looks like the last month. The demo that impressed everyone becomes the pilot that never quite earns its way into production, because 78% isn't good enough to trust unsupervised and it isn't improving on its own.

The near-universal reaction is to blame the model. Wrong model, wrong prompt, needs fine-tuning, wait for the next release. So teams swap models and re-engineer prompts and buy the upgrade, and the wall doesn't move, because the wall was never the model. The wall is that nothing in the system is learning. A frozen model at 78% will sit at 78% forever, no matter how good the 78% is.

The wall is a missing loop, not a weak model

Here's what a stalled pilot is actually missing, and it's two things, not one.

First, there's no eval harness. Nobody can say, with a number, whether a change made things better or worse. "It feels better" is not a measurement. Without a harness that scores the model against a real, versioned set of cases, every tweak is a guess, and you can't climb a curve you can't measure. Most teams that think they have a model problem actually have a measurement problem — they literally cannot see the curve, so they can't tell whether they're moving up it.

Second, there's no correction loop. In every real ops flow, a human is already fixing the model's mistakes — overriding a bad call, correcting a misclassification, rejecting a wrong extraction. That's a supervisor signal, and in a stalled pilot it evaporates. The human fixes the case, the case ships, and the correction is gone. The single most valuable training signal you own — a domain expert telling you exactly where the model was wrong and what right looks like — is being thrown away thousands of times a day.

What the curve looks like when the loop is closed

Put those two pieces in — a harness that measures and a loop that captures corrections — and the flat line stops being flat. It climbs.

That climb isn't a model swap. It's the same model, getting better because every supervisor correction becomes a training example and the eval harness confirms, week over week, that each change is real and not noise. On the Qrambo platform the observed rate is roughly +1.4 points per week off a flat 78% baseline — not from a smarter model, but from a system that stops throwing away what its operators already know.

Do the arithmetic and the strategic picture flips. A flat 78% is a dead end; you're stuck forever. A curve rising +1.4 pts/week crosses 85% in about five weeks and closes on the 90s inside a quarter — not because you bought anything, but because you kept the corrections your team was making anyway. The gap between those two futures isn't the model. It's whether the loop exists.

The threshold is what makes a rising curve safe to deploy

Rising accuracy is only half of what makes this deployable. The other half is knowing which cases to trust the model on today, at whatever accuracy it's currently at — and that's a confidence threshold, not an accuracy number.

Here's why the threshold matters more than the headline accuracy. The model isn't equally confident on every case — it's very sure about the routine ones and genuinely uncertain about the hard ones. Set a confidence threshold and you route by certainty: high-confidence cases clear automatically, low-confidence cases go to a human. That means even at 78% you can safely automate the slice the model is sure about, while every uncertain case — which is exactly where the mistakes cluster — lands in front of a person.

And the two mechanisms compound. Every case that crosses the threshold to a human is a case the human will decide on, and that decision is a correction that feeds the loop and lifts next week's curve. The threshold keeps you safe today; the corrections make you better tomorrow; and as accuracy climbs you can raise the threshold's automated share with evidence, not hope. That's the whole machine: measure, route, capture, repeat.

A worked example: the same case, two systems

Picture one misclassified invoice. An operator catches it, fixes the coding, and the payment goes out correct. In a stalled pilot, that's the end of the story — the case is right, the operator moves on, and the model has learned nothing. Next week an identical invoice arrives and the model makes the identical mistake, because nothing about last week's correction reached it. The operator fixes it again. And again. The human isn't a teacher here; they're a permanent patch, applied fresh every single time, forever.

Now run the same case through a system with the loop closed. The operator makes the same fix — same effort, same click. But the correction is captured as a labeled example and folded into the training signal. The eval harness scores the next version and confirms the model now handles that invoice type correctly, without regressing on anything else. The identical invoice next week clears automatically. The operator's one correction retired an entire class of future mistakes instead of patching a single instance of it.

That's the entire difference between a pilot that plateaus and one that climbs, and it's not a difference in the model. It's a difference in whether the system treats a human correction as disposable or as an asset. Same people, same corrections, same domain expertise being spent either way — the only question is whether any of it accumulates.

What it takes to actually build the loop

None of this is free, which is exactly why so many pilots skip it and stall. Closing the loop means three concrete pieces of infrastructure that a demo never shows and a from-scratch build usually defers.

You need a case store that captures every operator override with its full context — the input, the model's output, the human's correction, and the reason. You need an eval harness that runs candidate model versions against a versioned set of those real cases and produces a number you can trust, so "did this change help" has an answer instead of a vibe. And you need a retraining path that turns accumulated corrections into an improved model on a regular cadence, rather than waiting for a heroic quarterly fine-tuning project that never quite gets scheduled.

Miss any one of the three and the curve flattens. Capture corrections but never retrain, and the signal piles up unused. Retrain but skip the harness, and you're shipping changes blind, unable to prove you didn't make things worse. The three pieces only work as a set — which is why the honest version of "we'll build our own" is a lot more than a model and a prompt.

Why this is the part nobody demos

Vendors demo the model. They show you the shiny 78% and imply the rest is a matter of time. What they don't show — because it's unglamorous plumbing — is the eval harness and the correction loop that turn a static number into a rising curve. That's the supervisor-corrections moat: the corrections your operators generate in your domain are an asset a competitor can't buy, and a stalled pilot is one that never started collecting it. It's also why pilots stall at the eval-harness gap — without measurement, there's no way to prove a change helped, so nobody ships the changes that would move the curve.

If you're staring at a pilot parked in the high 70s, the useful question isn't "which model should I switch to." It's "where do my operators' corrections go, and can I prove whether last week's change helped?" If the answer to the first is "nowhere" and the second is "no," you don't have a model problem — you have a missing loop, and the loop is buildable. The product is built around exactly this: measure the curve, route by confidence, capture every correction, and watch 78% stop being a wall and start being a starting line.