Overview
Paste a git diff; Llama 3.3 on Workers AI checks it against your team's rules and returns pass, fail or incomplete, with rule, file and line for each finding. Rules are managed in chat, and every rule-changing tool call needs human approval.
- Role
- Sole author. Design-led, AI-assisted implementation with a public prompt log.
- Timeframe
- Sep 26 to Oct 2, 2026, the bulk built over 4 days
- Status
- Live demo on a custom domain; MIT-licensed; 44 commits
- Quality
- 561 unit tests across 25 files, all CI runs green; gitleaks, Semgrep and SHA-pinned Actions in CI; a post-deploy smoke script
Stack
- Cloudflare Workers
- Durable Objects
- Workflows
- Workers AI (Llama 3.3 70B)
- Agents SDK
- Workers Rate Limiting
- Vercel AI SDK v6
- Zod 4
- React 19
- Vite
- Vitest
Context
Teams want automated feedback on whether a change follows their own engineering standards. The hard part isn't calling an LLM. It's that the model reads untrusted input (diffs, user-written rule text) and must not be able to weaken the guardrail itself.
“The model can be fooled; it just can't do anything when it is.”
What it does
- Diff review. Validates the diff, skips lockfiles and binaries, flags minified or generated files, splits it into token-budgeted chunks, reviews each chunk with a tool-less model in JSON mode, validates every finding, de-duplicates, sorts by severity, and returns pass, fail or incomplete.
- Deterministic secret scan. 8 key-format regexes run over every added line, including skipped files. Matches become findings and are masked before storage.
- Rule management in chat. List, add, remove and restore rules. Each rule-changing call shows an approval card, and switching off a recommended rule needs a reason in the user's own words.
- Workspaces as capability URLs.
/w/{uuid}links; no accounts. - Free-tier cost control. Per-IP rate limits (IPv6 keyed by /64), a per-day neuron budget in a Durable Object with reserve-then-settle accounting, and a live “shared AI budget today” meter.
Architecture
Edge
Browser
Chat and a review box, over a WebSocket
Worker
Route allow-list
Callable from the review box
Agents
ChatAgent
Durable Object with SQLite: rules, chat, reviews. The chat model holds the rule tools and never sees a diff.
Trust boundary
ReviewWorkflow
Split, review chunks, combine, with retries. The review model has no tools and answers in JSON.
Every model call reserves budget first
Budget
NeuronBudget
A Durable Object per UTC day: reserve, then settle
Key decisions
Six of the 35 recorded decisions, each with the cost it accepted.
Remove the capability instead of detecting the attack
- Why
- The review model has no tools, diffs never reach the tool-holding chat model, and only validated
ruleId/file/line/verdictflow back. Detection is probabilistic; removing capability is structural. - Cost accepted
- The review can't act on its own findings; changes go through chat and approval.
Fail closed
- Why
- Return incomplete whenever a chunk fails, a finding fails validation, the cap is hit or a minified file is skipped. Severity comes from the rule, not the model. “Retry until valid” loops forever on a repeatable failure.
- Cost accepted
- More incomplete verdicts.
Line numbers the model copies rather than computes
- Why
- The validator checks that every cited line exists in that file's chunk.
- Cost accepted
- Slightly larger prompts.
One model config drives the token budget
- Why
- 24k context = 3k output + 1k system + 4,050 rule reserve, giving 12k chunks. Diffs of up to 18k tokens always fit in at most 2 chunks, checked by a seeded property-style test.
- Cost accepted
- Hard caps: 50 KB diffs, 50 rules.
Only an approved execute writes
- Why
- A rule changes only if the latest message approves that exact call ID and the call isn't in a server-side record of unapproved calls; valid rule IDs are a per-turn
z.enumin the tool schema. Born from a live loop on the demo. - Cost accepted
- Extra round-trips on refusals.
Observability deliberately off
- Why
- Workspace links are credentials, and logs would store them.
- Cost accepted
- Debugging only via live tail.
Deep dives
Finding my own bypass
- Skipping minified files protected the token estimate, but padding a line past 1,000 characters or naming a file
*.min.tshid SQL injection and still returned pass. I found it in review, wrote the bypass diff as a regression fixture, and changed skipping to force incomplete and name the file.
A live approval loop, and “only an approved execute writes”
- Incident
- On the deployed demo, “disable all rules” made the model alternate two refused calls. Each approval started a new turn, so the step cap never fired, and 8 approvals with a model-invented reason used more than 10% of the day's chat budget.
- Fix
- Over four iterations I made writes possible only inside an approved execute, required the reason to come from the user's own messages, batched changes into one card, and encoded valid rule IDs in the schema. Every live incident became a replay test.
- Outcome
- Switching off four rules went from 4 approvals and ~8 model calls to one call and one card (measured live on the demo).
A duplicated-stream bug in a dependency
- Workers AI emitted each delta twice per chunk; the provider read both, doubling text and corrupting tool-call JSON. Instead of patching the dependency, I wrapped the AI binding in a
TransformStreamthat drops the duplicate copy. Tests replay three recorded streams split down to 1-byte pieces, and one test pins the upstream bug so the wrapper can be removed once it's fixed upstream.
Running a public LLM demo on a free tier
- Reserve-then-settle accounting lives in a per-day Durable Object (synchronous SQL, so two reviews can't both take the last of the budget), idempotent by review ID; IPv6 rate keys use the /64. I measured chat cost per turn before choosing, and reverted “rules in the system prompt” when it cost more and put untrusted text in the system prompt.
How it was built
Design first: a design doc, a decision log and agent instructions were committed before any code. Implementation was AI-assisted (Claude Code) in reviewed steps, with every prompt logged publicly in the repo; one core module (resolveRules) was hand-written. I made every design decision, found the bypass in review, specified the security mechanics of the fixes, and turned each live failure into a test.
Numbers
- Unit tests passing in CI across 25 files
- 561
- Recorded design decisions (context, options, decision, consequences)
- 35
- Trust-boundary layers in the design doc
- 7
- Secret-format detectors
- 8
- To switch off 4 rules, down from 4 approvals and ~8 model calls (live demo measurement)
- 1 call, 1 card
Limits
- No authentication: workspace links are bearer credentials and can't be rotated.
- A demo, not a multi-tenant product; measurements are small development samples.
- Prompt-injection resistant by design, not injection-proof.