Production 101 - #16 QA's Role in the Production Process
What quality assurance actually does, and how to use it as a production signal
Game Production Alchemist › Production 101 › Game QA Role in Production: QA vs QC Explained
Most studios call their testers “QA” but run QC, and understanding the difference changes how you use the function.
Involving QA leadership before development starts is more effective than involving them after the build arrives.
Bug density, reopened bugs, and first-pass pass rates are signals about your project, not just your test coverage.
QA leadership often has better real-time visibility into project health than anyone else on the team.
This is part of the Production 101 series.
QA finds bugs. Everyone knows that. What most producers don’t know is that QA also tells you the truth about your game at the point when everyone else has developed an interest in the answer being favourable. An honest QA report, read correctly, is one of the most useful documents on a project.
QA vs. QC: the distinction that matters
These two terms get used interchangeably in almost every studio I’ve worked in, and the confusion causes real problems. So: QC is Quality Control. It’s the act of finding defects. A tester runs a build, finds bugs, files them. That’s QC. It’s valuable, it’s necessary, and it’s what most game studios are actually running when they say they have QA.
QA is Quality Assurance. It’s the broader discipline of building processes that prevent defects from reaching the build in the first place. That means reviewing requirements before development starts. It means writing acceptance criteria so the team agrees on what “done” looks like before anyone writes a line of code. It means defining test plans early, participating in iteration planning, flagging ambiguity in design documents when there’s still time to address it without cost. The invisible work in QA is what makes the visible work of QC easier.
The distinction matters because the timing is completely different. QC finds problems late, after they’ve been built. QA prevents some of them by catching the conditions that produce them before they’re built. Both happen in a well-run studio. Many studios have only QC and call it QA, which means they’re discovering late what they could have prevented early.
If you only have QC, you’re finding problems when they’re expensive. If you have QA, you’re preventing some of them when they’re cheap.
There’s a management habit that makes this worse: treating QA as a downstream service. Development builds something, then throws it over the wall to QA for testing. That model produces exactly what you’d expect from it. The QA team gets builds they haven’t seen specifications for, against requirements they weren’t part of writing, and they find the same kinds of problems at the same stage of every project. The producer wonders why testing always takes longer than planned. It takes longer because the process was designed to find problems late.
The studios I’ve seen run this well treat QA as a discipline that spans the full development cycle, with dedicated headcount that has time for both QC work in the build and the upstream process work that prevents some of it. That combination isn’t free. It requires the producer to protect QA’s time for non-testing work, which is harder to justify when there’s always a build that needs testing.
Shifting left
“Shift left” has become one of those phrases that gets repeated in planning meetings without anyone doing the thing it describes. So let me be specific about what it actually means and why it works.
Shifting left means moving testing activity earlier on the project timeline, closer to when decisions are made. A QA engineer involved in iteration planning reviews the acceptance criteria for the iteration’s work before development starts. They ask the question: how will we know this is done, and done correctly? That question, asked before development, exposes ambiguity that would otherwise surface as a bug after development. The design decision that seemed clear to the developer and the designer doesn’t look the same from the QA perspective, and that gap is exactly where defects live.
The practical implication for a producer is this: QA leadership should be in iteration planning. They should be reviewing acceptance criteria before the iteration begins. They should have a seat at the table when requirements are written, not a folder of requirements handed to them after the fact.
This is a specific operational choice that costs something. QA engineers in planning meetings are not testing the current build during that time. The payoff is a build with fewer ambiguity-driven defects in it, which means QC work goes faster, which more than covers the time spent. In my experience, the studios that resist this do so because they’re measuring QA by test execution metrics and those metrics look worse when testers are in planning meetings rather than executing test cases. The wrong measurement produces the wrong behaviour.
I’ve pushed for QA involvement at planning on several projects where the resistance was real. The most common objection is “QA is already stretched.” That’s usually true. And it’s usually true because QC is running at full capacity catching problems that earlier involvement would have prevented. Breaking that cycle requires trusting the investment before you can measure the return.
QA as a production signal
The data QA generates is a map of where your project’s risks are materialising. Most producers read bug reports as a testing problem. I read them as a production signal.
Bug density by area tells you where the team is struggling. If QA is consistently finding more bugs in one system, in one level, in one feature area, that’s telling you something about that part of the codebase or design. It might be that the area is technically complex. It might be that the requirements were unclear. It might be that a specific developer is under pressure and cutting corners, or that a handoff between two teams isn’t working. The bugs are the symptom. Your job is to read the pattern.
Reopened bugs tell you where estimation went wrong or where fixes were rushed. When a bug gets fixed, verified, and then reopens because the fix introduced a new problem, you’re looking at one of two things: the fix was incomplete because the developer didn’t understand the root cause, or the fix was done too quickly because the schedule didn’t leave time to do it properly. Either is worth understanding.
First-pass pass rates tell you about build quality. If your QA team consistently finds that a high proportion of new features fail their initial testing, that’s a signal about the quality of what’s coming out of development, and that’s a conversation to have before the next iteration.
The bugs are the symptom. The production question is what the pattern tells you about how the team is working.
None of this replaces reading the actual bug report. It means you’re reading it differently. The question to ask is what the distribution tells you, and where the project needs attention.
Test plans and milestone acceptance
This is where QA connects directly to the legal and commercial structure of the project. Production 101 #8 covers the milestone structure in contracts. The short version: milestones in a development agreement carry acceptance criteria, and those criteria determine whether the milestone gets paid. The question of who writes those criteria, and what they actually test, is where test plans become contractually significant.
The acceptance criteria problem is this: “done” means different things to the development team, QA, and the publisher, and the place these three definitions have to match is in the test plan. The development team thinks done means the feature is implemented. QA thinks done means the feature passes the test cases. The publisher thinks done means the feature meets the specification they signed off. Those are often different things. The producer’s job is to find out they’re different before the milestone submission, not after.
This is why QA involvement in milestone planning matters, not just milestone testing. If the QA team isn’t part of defining the acceptance criteria for a milestone, you’ll have a test plan that tests the wrong things, and a milestone review where the publisher’s criteria and your test plan don’t match. That’s a slow, expensive conversation to have at the wrong moment.
The test plan is also your record. In a milestone dispute, the test plan is the document that says what was tested, how, and with what outcome. A well-structured test plan that maps directly to the milestone acceptance criteria is evidence. A set of bug reports with no structure is noise.
The QA/producer relationship
QA leadership often has better real-time visibility into project health than anyone else on the team. Think about what they see: every feature at the point of completion, every bug through its full lifecycle, every area of the game that’s being tested and retested. They see the fragile systems before they become critical. They see the features that are repeatedly failing verification before that becomes a milestone problem. They have a picture of the project that no one else has, and they have it before anyone else.
Building a genuine working relationship with QA leadership means getting that picture early. That requires two things. First, the QA lead needs to trust that honest reporting will be used to solve problems, not to assign blame. If the first time a QA lead tells a producer “this area is fragile and we’re worried about it” the producer responds by escalating to leadership and making it a crisis, the QA lead will learn not to tell the producer that in advance. The honest reporting that makes early warning possible depends on the producer responding to it well.
Second, it means the producer has to actually ask. A QA lead who isn’t being asked will tend to report what’s in the system: bug counts, coverage percentages, test execution status. Those metrics are accurate and not very useful. The useful information is the qualitative read: which areas feel stable, which feel fragile, where are the testers finding unexpected behaviour that doesn’t have a bug filed for it yet. That information requires a conversation, not a dashboard.
I’ve had QA leads who’ve become the most reliable early warning system on a project, and I’ve had QA leads I barely spoke to because I was managing them as a service function. The difference in what I knew about my projects in those two situations was significant.
Live service QA specifics
Regression testing on a live game is a different problem from certification testing on a boxed product, and the difference goes beyond scale.
On a boxed product, you’re testing toward a fixed release. The risk is a defect in the shipped build. On a live game, you’re testing toward a continuous stream of updates against an existing player base. A broken feature on a live game affects your current players immediately, and the moment the update is live, you can’t take it back without a hotfix. That changes the risk calculation completely.
The regression scope question is one of the harder ongoing decisions in live service production. You can’t regression-test everything every time you ship, because the cadence of a live game doesn’t give you the time. What you can do is maintain a regression suite that covers the highest-risk areas: systems that have broken before, systems that many players interact with, monetisation flows. The question of what’s in that suite and when it gets updated is a joint decision between QA and production.
Hotfix protocols matter more in live service than anywhere else. When something breaks in production, the pressure to fix it quickly is intense. The risk is that a rushed hotfix introduces new problems. The discipline of having a fast-track process that still includes a meaningful QA pass, even if compressed, is what prevents one bad deploy from turning into two. That process needs to be designed before something goes wrong. Designing it during the incident is too late.
The “good enough to ship” question gets asked constantly in live service. The answer requires both QA data and production context, and the producer has to own the call.
The judgement call of when a risk is acceptable and when it isn’t is one of the hardest things about live service production. QA can tell you what’s broken and how severe it is. They can tell you what the workaround is, if one exists, and what the likelihood is of the bug being encountered in normal play. The producer has to add the context: how many players are affected, what’s the competitive window, what’s the cost of delay. That combined assessment is the decision. Neither side can make it alone.
If you want context on how milestone structures connect to the contractual picture, Production 101 #8 on legal documents covers the framework. For how the producer’s reporting picture relates to what QA is generating, Production 101 #9 on status reports is the relevant entry.