The Artefact Arrived Before Anyone Was Ready to Review It
AI didn’t create the evaluation problem. It made ignoring it impossible to justify.
Shift-left testing was coined in 2001. AI teams are discovering it the hard way in 2026.
A polished artefact arriving fast suppresses the review instinct in senior stakeholders.
AI multiplies transmission volume without expanding reception capacity.
Producers don’t just manage deliverables. They manage the channel.
Cristiano Pierry published a thoughtful piece last week arguing that evaluation is becoming a product surface, not a back-office activity. He’s right about the problem. I want to push on the diagnosis.
The evaluation problem isn’t new. What’s new is the rate at which AI is making it impossible to ignore.
If this is useful, there's more where it came from. Multiple free posts a week on game production, applied AI, leadership and how the sausage actually gets made.
Larry Smith coined the term “shift left” in a 2001 article in Dr. Dobb’s Journal. The idea is simple enough to fit on a Post-it: find defects earlier in the development lifecycle, because defects found late cost more to fix and cause more damage. Move quality gates left on the timeline, toward the work, rather than leaving them clustered at the end.
Game studios learned this expensively. I lived through the era when QA was something that happened after production, when the build was “done.” It wasn’t done. It was never done. It was just at a point where the team had run out of time to keep making it better. The bugs were always there. The only question was whether we found them in a sprint review or in a launch post-mortem.
The discipline eventually moved. Studios got better at involving QA earlier, at writing acceptance criteria before implementation, at treating a failed test in week three as cheaper than a failed launch in week forty. It required culture change, and culture change in studios is slow, but the direction was correct and the reasoning was sound.
What Pierry is describing, the need for evaluation to be continuous and structural rather than deferred and ritual, is shift-left applied to AI product work. The insight is good. The reason it keeps needing to be rediscovered is worth examining.
The artefact is why.
In older workflows, the artefact moved slowly. A design review happened while sketches were still rough. A requirements review happened while the spec was still being written. The incompleteness of the thing in front of you was visible, and visibility triggers the review instinct. When something looks unfinished, people know to bring judgment to it.
AI collapses that gap. The first output is fluent and polished. It has the texture of something that has been through several iterations. It sounds confident. The UI renders. The copy flows. The prototype responds.
That texture short-circuits the review instinct in the people who most need to apply it.
A fluent artefact does not mean a sound one. But it feels that way in the room, and feelings drive process.
Senior stakeholders in a position of approval have a pattern I have seen repeatedly across studios and clients: they do not take early drafts seriously. They want to see it in situ. They want to see it on device. They resist forming a view until the thing feels real, which usually means until it looks finished.
This was already a problem before AI. With AI, it becomes a crisis, because the thing always looks finished. The prototype that takes forty minutes to build looks, to a distracted VP with fifteen minutes in their calendar, indistinguishable from the prototype that took four months. The review instinct never fires. The evaluation layer gets skipped.
Then ten more artefacts arrive the next day.
Shannon’s channel capacity theorem, first published in 1948, describes the theoretical maximum rate at which information can be reliably transmitted over a communication channel given its bandwidth and signal-to-noise ratio. Exceed the capacity, and you don’t get a little degradation. You get overflow. The signal doesn’t slow down gracefully. It stops being received correctly.
Communication has two sides. Transmission and reception. Both sides matter, and they are not symmetrical.
AI has made the transmission side of product work effectively infinite. The cost of generating a prototype, a spec, a set of test cases, a visual variant, a conversation path has dropped so far that volume is no longer a practical constraint. A team that used to produce three artefacts a sprint can now produce thirty.
Thirty-plus years of production work went into this. Most of it didn’t make the post. New posts every week. Free to read, free to subscribe.
The reception side has not changed. Senior stakeholders have the same number of hours they always had. Their ability to process, evaluate, and form a judgment on what they are seeing has not increased. Their attention is a finite channel with a fixed capacity.
What happens when you transmit more information than the channel can receive? The buffer fills. The signal overflows. The receiver stops processing correctly, not because they are incompetent or disengaged, but because the incoming rate exceeds their capacity for reception.
This is not a stakeholder problem. It is a systems problem. And it is a producer problem.
Producers are not just deliverable managers. They are communication channel managers. The job has always involved monitoring both sides of every information exchange: are we transmitting clearly, and is the receiving party actually receiving?
In practice, this means pacing. It means knowing that a stakeholder who has just reviewed six artefacts this week has a degraded capacity to usefully engage with a seventh. It means understanding that sending more is not the same as communicating more. It means recognising that an unanswered email chain with twelve attachments is a channel in overflow, not a team that is working hard.
The temptation with AI is to celebrate throughput. The discipline is to regulate it.
A team that uses AI to generate thirty artefacts and submits them all for review has not improved its output. It has created a reception problem. The evaluation that gets skipped is not a stakeholder failure. It is a production failure. Somewhere, someone chose to maximise transmission without accounting for what the receiving end could handle.
Shift-left does not solve this on its own. Moving the evaluation gate earlier helps when you have one artefact and a willing reviewer. When you have thirty artefacts and a reviewer whose buffer is already at capacity, shifting left just means the overflow happens sooner.
The producer’s job is to manage the flow rate. Decide what goes through the channel and when. Batch thoughtfully. Sequence for attention, not for volume. Give stakeholders the right thing to review at the right time, in a form that makes the review instinct fire rather than suppress.
Pierry’s framing, that evaluation is becoming a product surface, is a useful provocation for teams that have not thought about it at all. For producers who have been doing this for any length of time, it is a restatement of something we already know: quality gates belong in the work, not at the end of it.
The part worth adding is this: when you can generate artefacts faster than your stakeholders can evaluate them, the constraint is no longer production. It is reception. And managing reception has always been part of the job.
The channel has a capacity. It always did. AI just made it possible to exceed that capacity so fast that no-one notices until the reviews stop happening, the sign-offs go quiet, and the artefacts keep coming. That is not a stakeholder attention problem. It is a production problem. And fixing it is the producer’s job.
If this changed how you think about even one thing, the next post might too.