Launch Audit Kit
A pre-launch audit crew — 18 agents, a Playwright harness, a red team per agent and a cross-agent reviewer
Problem
Auditing a project end to end before launch is the easiest thing to skip in a one-person team. "I had a look" is not reliable, because the checklist lives in your head and different items fall off it on every project. The second issue is subtler: telling a language model to "audit this site" does not work. A finding that is not grounded in measurement comes out confident and wrong, and its wrongness only becomes visible when you try to fix it. The value of an audit is not the number of problems found but the fact that every problem found is real.
Users
A single user: me, on my own projects, before launch. Projects are not at the same maturity — some are early prototypes, some are live systems — and the same checklist does not fit both. That is why the pipeline profiles the project before the audit starts.
My Role
The whole system is mine: agent definitions, harness scripts, the install script and the report format. The kit is installed by copying it into a project and runs inside that project.
Architecture
The pipeline runs in order. A scout profiles the project and decides whether the conditional auditors are needed. Then the Playwright harness runs in parallel: screenshots, a three-run Lighthouse pass and an axe accessibility scan. For projects at production maturity, a real build output is taken as well. Once those measurements exist, 13 fixed and 1 conditional auditor run in parallel — visual design, interaction, conversion, copy, design system, technical and content SEO, frontend and backend performance, accessibility, security, legal compliance, data integrity, and data visualisation if it is detected. Each auditor's output goes into its own QA red team; findings that survive go to a single cross-agent reviewer. The result is a markdown report with an approval checkbox next to every finding. A single write-capable agent applies the approved fixes, and a verification pass follows.
Key Decisions
The audit starts after measurement; it does not replace it
Before the auditors say anything, the Playwright screenshots, the median of a three-run Lighthouse pass and the axe output are already in hand. A performance finding is no longer "the page feels slow" but is written against a measured metric. The same applies to accessibility: a violation caught by axe and a model's guess do not carry the same weight.
Trade-offSetting up the harness is an extra step on every project and the route configuration is written by hand. On a project where measurement is not possible — an application that does not come up locally, or needs a database — the pipeline narrows and part of the audit simply cannot run.
Every auditor's output passes through a separate red team
Having one agent review its own finding does not work; it defends its own text. So each auditor has a separate QA pass whose job is to find fault: hallucination, severity inflation, blind spots. A finding that does not pass is sent back at most twice.
Trade-offThe agent count, and therefore the cost, nearly doubles and the audit takes longer. On a small project that cost can exceed the value of the problem found — which is why I run the kit on projects that are about to launch, not on every project.
Write access belongs to one agent, and its scope is fixed in advance
None of the agents that produce findings can modify a file. Only the fix-applier applies changes, and it cannot step outside the file list of the approved fix. That way, "something broke during the audit" never turns into a hunt for which agent did what.
Trade-offAn obvious problem outside the scope is left unfixed and needs a separate pass. Because bulk fixes all go through a single agent, the pipeline bottlenecks here.
No fix is applied without approval
The report arrives with markdown checkboxes: apply, defer, won't fix. The decision stays with me, because some findings are correct but wrong for that project — a legal-compliance finding on an early prototype is technically right and practically not the next task.
Trade-offOn a hundred-item report the approval step can take as long as the work itself. Bulk approval shortcuts soften this but do not remove it; it is not a fully automatic pipeline, and I do not want it to be.
Stack Rationale
The agents run on Claude Code; the measurement side is deliberately outside the model: Playwright, Lighthouse and axe. That separation is the backbone of the pipeline — the model's job is to interpret and prioritise, not to measure. The harness is TypeScript, because most of the audited projects are Next.js anyway.
Security
The kit only runs inside the project it was copied into. The overwhelming majority of the agents are read-only; the single write-capable agent is limited to the file list of an approved fix. Audit output stays in the repository and is not sent anywhere.
Performance
Lighthouse runs three times and the median per metric is taken — a single run, especially on a cold start, does not represent reality. The auditors run in parallel, so the pipeline's total duration depends on the longest chain rather than the slowest single auditor.
Results
- 18 agents · 5 slash commands · 5 TypeScript harness scripts
- 13 fixed auditors + 1 conditional (when a dashboard or chart library is detected)
- A red team per agent plus a single cross-agent reviewer
- Write access belongs to one agent, and it cannot step outside the approved file list
What I Would Do Differently
I should have added the cross-agent reviewer from the start. In the first version every auditor wrote its own report, and the same problem arrived from three different agents with three different severities; whoever read the report did not know which one to trust. The same pattern shows up on the code side of this portfolio: in Bergaz Operations, leaving the guarantee in the application layer meant every new path had to re-establish the same rule. Here too, until deduplication moved into one place, every auditor was doing its own.