Beyond copilots: designing AI workflows that stick

You bought the licenses. Adoption looks fine on the vendor dashboard. A year later, nobody can name a single number that moved.

I hear this constantly, usually in a slightly embarrassed tone, as though the company failed a test everyone else passed. It did not. The rollout did exactly what it was designed to do, and what it was designed to do was never going to show up in your metrics.

Here is why, and what to do instead.

The arithmetic nobody ran before signing

A copilot makes one person faster at one step. Usually the step where they write something.

So do the sum. If writing code is 20% of the time between a ticket being ready and that work being live, and you make that 20% twice as fast, you saved 10% of the cycle. Real, but invisible. It disappears into the noise of a normal week.

The other 80% is waiting. Waiting for a review. Waiting for someone to explain what the ticket actually meant. Waiting for a test environment, an answer from a stakeholder, a deploy window. None of that got faster, because none of it was what you bought.

That is the whole story of most Copilot rollouts. You optimized the cheap part.

Hold on to two items from that waiting list: the explaining and the reviewing. That is where the gain turned out to be.

There is a second problem stacked on top. A license is a private tool. It makes individuals faster in private, in their own way, with their own prompts, at their own standard. Twenty engineers with twenty personal setups is not a capability. It is twenty experiments nobody is comparing.

What we replaced it with

We stopped giving people chat windows and built one shared harness instead. Every engineer works through the same one. This is the full description, because the details are the point.

It lives in the repo, so it is versioned like code and reviewed like code. It carries the architecture rules, the test conventions, the code standards, and the definition of done. When someone improves the setup, everybody gets the improvement in the next pull. Nobody maintains a private prompt library.

It runs in three phases, and writing the code is the middle one. That is the entire reason it worked.

The first phase is product work, done with the product manager rather than for them. They sit with the agent and shape the thing: what it is for, what it has to do, which edge cases matter, what is explicitly out of scope. The output is a state file, and that file gets committed to the repo next to the code it describes.

That single detail changes more than it looks like it should. The specification stops being a memory, a meeting, or a comment thread somebody has to reconstruct on a Tuesday. It becomes a versioned artifact that every later phase can read, that every engineer can open, and that shows who changed what and when. The explaining does not have to happen again, because it happened once and got written down in a place the machine can also use.

The second phase is the build, and it is the least interesting part. The agent works from that state file, plus the Jira ticket and its comment thread, plus live Datadog error rates, traces, and logs for the service it is about to touch. It does not need re-briefing. Everything it needs is already written somewhere it can read.

The third phase is review, and it is split. The engine takes the first pass, the mechanical and tedious one that humans skim when they are busy. A human still approves every time, and we are not planning to remove that. But because the boring pass is finished before a person opens the pull request, human review became extremely fast. The reviewer spends their attention on whether this is the right solution, not on whether someone forgot a null check.

So the unit of work is no longer "write this function". It is "take this problem to something a human is happy to approve".

What it did, including what it did not

Development speed went up 4x. Bugs went down 28%.

The second number is the one that pays. Fewer bugs means less rework, and rework is the cost that never appears in anyone's plan because it hides inside the next sprint. Speed alone would have just produced defects faster.

Look at where that speed actually comes from, and it is not the typing. It is the waiting we went after. Ambiguity that used to surface in week two now surfaces in the first hour, because writing the state file forces the question while the product manager is still in the room. Reviews that used to sit in a queue for a day now take minutes.

The honest limit is this. A harness cannot make anyone want to decide. What it does is make the cost of not deciding visible immediately instead of a fortnight later, because a product manager who will not commit to what a feature is for now produces a state file that says exactly that, in writing, with their name on it. Some people find that uncomfortable. I think that is the point.

The gain shows up as things you stop outsourcing

This is the part that surprised me, and it is the part worth telling your board.

When a team gets meaningfully faster and more reliable, the interesting change is not that the same roadmap arrives earlier. It is that work you had written off as too expensive comes back on the table.

We brought search in-house. It had been running at 2.45 seconds. It now runs under 300 ms, and it takes roughly $200,000 a year off the bill that used to go to a vendor. Before, that was a project nobody would have approved. We built an order management system in two months with two developers, and review scores went from 3.6 to 4.2 stars. Same reason: it stopped being unthinkable.

We also put the data lake behind Slack, with bronze, silver and gold layers underneath, so people ask questions in a channel and get answers. The analyst queue that used to sit in front of every decision is gone.

None of those are copilot wins. They are capacity wins, and capacity is what a redesigned workflow buys you.

The hardest part has no technical component

Redesigning the work changes what people are for, and that is where these programmes actually die.

An engineer who spent fifteen years being the person who could write the tricky part now gets judged on how well they specify and how sharply they review. That is a real loss for some people, and treating it as a training issue is how you lose your best ones. Say it out loud instead: the craft moved, it did not disappear, and reviewing well is harder than writing well.

Some people never take to it. That is allowed. What is not allowed is pretending the change is purely a tooling upgrade while quietly expecting everyone's job to be different by Q3.

Then measure it, or you are just guessing

Everything above is a claim until you can show it. Before scaling any AI workflow, you need a way to tell whether the output is good, one that survives a bad week and does not depend on whoever demoed it being in the room.

That means evals: a fixed set of real cases, run every time you change the prompt or the model, scored the same way each time. It is less work than it sounds, and it is the whole subject of the next post, so I will leave the how there.

Start here this week

Take one workflow. Not the whole company, one workflow. Time a single piece of work from ready to live, and mark every hour as either working or waiting.

You will almost certainly find that the part you bought a licence for was never the expensive part. That measurement is worth more than another round of tool evaluations, and you can have it by Thursday.

Then redesign the biggest waiting block, and only then decide where AI belongs in it. The goal was never to use AI everywhere. It is to use it exactly where the payoff beats the risk.

If you want a hand doing that properly, that is the work I do.