Layered AI code review — hooks, sub-agents, and a second model

The moment you decide “we’ll let AI review our PRs,” the options bloom: GitHub Copilot Review, CodeRabbit, cubic, Graphite’s Diamond, or wiring up your own sub-agent alongside a Codex CLI call. All of them are impressive in a demo, and all of them lead to the same conclusion after a few weeks in production — giving one LLM the whole review job silently fails. False positives pile up, engineers stop reading the output, and the review layer becomes ceremonial.

This is a field report on a different shape: layer the responsibilities. Deterministic gates before commit, a Claude sub-agent before push, and a second model (in our case Codex) for the milestone checks. Three tiers, each with a narrow job. Together they hold up long enough to be worth running. This is a companion piece to Making AI actually follow the rules with Claude Code hooks, where the Tier 0 mechanics are covered in depth.

Two reasons a single-model review collapses

Understanding why one-model review breaks matters, because it shapes what layers you actually need.

False positives destroy trust before you get to the true positives. Once the review layer feels noisy, engineers stop looking. That’s it — the layer is dead, and no amount of subsequent tuning brings the attention back. Absolute false-positive rates vary wildly by tool and configuration, so treat any single number with suspicion — Graphite’s guide cites a 5–15% range as an industry estimate, though without linking primary research.

The more useful evidence is Cloudflare’s published rollout data. Across the first 30 days of company-wide AI code review, they ran 131,246 reviews against 48,095 merge requests. The number worth internalizing is the output volume: they tuned down to 1.2 findings per review on average, and developer “break glass” overrides landed at 0.6%. Their own framing — “We biased hard for signal over noise” — makes the point that suppressing findings was itself the design decision, not a side effect.

No single model has full coverage. Every LLM has systemic blind spots. Claude is strong at long-range reasoning through a diff; the Codex line (GPT-5.x) is finer-grained on code-pattern inference. Bugs living in the blind spot of your chosen model will keep sliding through, indefinitely. The insurance is not “more prompting” — it’s “another model.” A second reviewer trained on a different mixture will catch a different slice.

The answer is to layer by responsibility.

Tier 0 — deterministic gates, every commit

Type checking, lint, format check, and secret scanning belong in a pre-commit hook, executed deterministically. No LLM here. Adding an LLM inflates latency, adds cost per commit, and injects flakiness where you can least afford it.

# .claude/hooks/pre-commit-check.sh (excerpt)

set -euo pipefail

# Secret scan (all repos)
if git diff --cached | grep -E "(sk_live|api_key=|PRIVATE KEY|ghp_[A-Za-z0-9]{36})" >&2; then
  echo "❌ secret pattern detected" >&2
  exit 2  # Claude Code hooks require exit 2 to block; exit 1 passes through
fi

# Stack-specific gate (TS / Node example)
if [[ -f package.json ]] && command -v pnpm >/dev/null; then
  jq -e '.scripts.typecheck' package.json >/dev/null && \
    CI=true perl -e 'alarm 120; exec @ARGV' pnpm typecheck || {
      echo "❌ typecheck failed" >&2
      exit 2
    }
  jq -e '.scripts.lint' package.json >/dev/null && \
    CI=true perl -e 'alarm 120; exec @ARGV' pnpm lint || {
      echo "❌ lint failed" >&2
      exit 2
    }
fi

exit 0

Speed is the whole point of Tier 0. Full builds (astro build, next build, etc.) are too heavy for every commit and belong in CI or the push gate. Test runs need to be scoped (changed files or a fast subset), not the whole matrix. If Tier 0 doesn’t fit inside 30 seconds, engineers will disable it, and no gate at all is worse than an imperfect gate.

Tier 1 — Claude sub-agent, on push

Right before git push runs, Claude Code’s experimental agent hook can spawn a sub-agent to review the diff. This is Tier 1. LLMs are inherently non-deterministic, so the block criteria must be conservative — block only what would clearly ship a bug, keep everything else advisory. Wire it in ~/.claude/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "if": "Bash(git push*)",
        "hooks": [
          {
            "type": "agent",
            "prompt": "Review the diff between the current branch and origin. Return {\"ok\": false, \"reason\": \"...\"} ONLY for high-confidence, high-severity issues in: (1) plaintext secrets or credentials, (2) missing auth checks, (3) SQL injection / XSS / SSRF, (4) obvious crash-inducing bugs. Everything else is advisory — return {\"ok\": true}. Do not report findings below confidence 75. If the latest commit message contains [skip-review], unconditionally return {\"ok\": true}."
          }
        ]
      }
    ]
  }
}

The first trap here is a permissions problem. The sub-agent needs to invoke git diff, git log, git show to read the diff at all. If those aren’t in permissions.allow, it fails with “cannot evaluate” and returns ok: false — every push blocks. That’s the correct default (fail-closed), but it only becomes correct once the permissions are actually granted. Do it once at user-global scope:

{
  "permissions": {
    "allow": [
      "Bash(git log:*)",
      "Bash(git diff:*)",
      "Bash(git show:*)",
      "Bash(git status:*)",
      "Bash(git rev-parse:*)"
    ]
  }
}

The second trap is the experimental matcher itself. if: "Bash(git push*)" occasionally misfires on compound commands like git check-ignore && git status — the parser treats the chain loosely. Simple git push never triggers this, but it happens often enough that having a fast [skip-review] bypass in the commit message is worth wiring in as a matter of hygiene.

Tier 2 — Codex as the second model, at milestones

Tier 1 alone is usable, but leaves you dependent on one model’s blind spots. Bring in Codex (OpenAI’s line) as a second reviewer, invoked manually at milestone points: before opening a PR, after a large refactor, before a release. Codex CLI ships a codex exec review subcommand that handles diff scope and review logic without a custom prompt (behavior below verified on 0.142.5).

# Review uncommitted work
codex exec review --uncommitted -m gpt-5.6-sol --json

# Review commits ahead of the base
codex exec review --base origin/main -m gpt-5.6-sol --json

# Review a single commit
codex exec review --commit <sha> -m gpt-5.6-sol --json

Pin the model explicitly with -m. As of July 2026, gpt-5.6-sol is what Codex’s own catalog labels the latest frontier agentic coding model; the gpt-5.6 alias resolves to the same thing. Pass --json for JSONL findings and filter to confidence ≥ 0.8.

The value of the second model isn’t finer inspection — it’s catching the class of bug the first model won’t see. When Codex flags something Claude missed (or vice versa), that’s evidence the layering is doing work. If they always agree, you probably have redundant reviewers, not layered ones.

Codex costs real money, so don’t run it on every push. In our setup Tier 2 is tied to an optional argument on /ready — /ready full or /ready codex — so it fires only at genuine checkpoints. Heavy layers run on demand, light layers run always. That rhythm is what makes the whole system livable.

Presenting the tiers as one gate

Three tools, three concerns, but from the developer’s seat this should look like one gate. A /ready slash command wraps the layers and prints a single table:

Check	Kind	Status	Findings	Blocks
astro check	deterministic	✅ pass	0 errors / 0 warnings	required
pnpm typecheck	deterministic	✅ pass	—	required
pnpm lint	deterministic	✅ pass	—	required
code-reviewer	LLM	✅ pass	🟡 1 (docstring rot)	advisory
secret-detector	LLM	✅ pass	—	advisory
codex	LLM (external)	➖ skipped	run /ready full	advisory

The key affordance is the visual split between “deterministic” and “LLM.” A red deterministic row means don’t commit. An LLM row is context — actionable if you agree with it, ignorable if you don’t. The confidence ordering (deterministic > high-confidence LLM > general LLM findings) becomes intuitive without needing to spell it out.

What to block, and what to leave advisory

The single biggest configuration decision in a layered setup is the block criteria per tier. Our current working rules:

• Block: deterministic failures. Type errors, lint failures, secret matches, dangerous commands.
• Block: high-confidence LLM findings on critical categories. Explicit missing auth, plaintext credentials, clear SQL injection / XSS / SSRF, obvious crash-inducing bugs.
• Advisory: everything else the LLM says. Design suggestions, readability, refactor opportunities, performance tuning, test-coverage hints.

We started with the tightest possible interpretation of “high-confidence, critical,” advisory’d everything else for four to six weeks, and only promoted categories to block after we saw the false-positive rate stay in the low single digits. This is the same instinct behind Cloudflare deliberately suppressing finding volume: aggressive block rules destroy trust immediately, and engineers who reflex [skip-review] today will keep doing it when a genuinely critical finding lands next quarter.

Hooks vs. on-demand: pick per check

Beyond tiering, there’s a second axis: hook vs. on-demand. Route to on-demand (a /ready invocation) when the check:

• Needs a sub-agent (hooks can’t call sub-agents directly);
• Takes more than a couple of seconds (delay at every tool call is unlivable);
• Represents “I want to check this now” (before a PR, before a release).

Route to hooks when the check is deterministic, sub-second, and belongs on every commit. Hooks host the fast deterministic gates. On-demand hosts the heavy LLM review. Trying to balance both across a single mechanism ends up compromising on both.

Feed all reviewers from the same repo rules

Maintaining separate prompts for the Claude sub-agent and Codex is a fast path to drift. Instead, keep the review rules in Markdown at repo root and treat both reviewers as consumers of the same source.

Codex concatenates every AGENTS.md from the repo root down to your working directory, with the closer files appearing later so they override earlier guidance. Note the gap that trips people up: splitting review rules into a separate file like code_review.md does not get picked up automatically — you have to reference it explicitly from AGENTS.md for it to apply during review. On the Claude side, pull the equivalent section from your CLAUDE.md chain and inject it into the sub-agent prompt.

When the rules live in Markdown, updates are a one-file PR. When they live in separate prompts, each reviewer subtly diverges, and the layered system quietly loses coherence.

Wrap-up

Handing your review pipeline to a single LLM is comfortable and short-lived. False positives will erode trust, and no amount of prompt polishing recovers a review layer that engineers have stopped reading. Layering — deterministic before commit, sub-agent before push, second model at milestones — is the shape that survives in production. Every layer has a narrow job. Every block criterion is deliberately conservative. The result is not a spectacular AI reviewer; it’s a boring, reliable one that keeps catching the class of bug it was scoped for.

Start with Tier 0 hooks (see the hooks piece for the implementation traps), add a thin Tier 1 sub-agent once permissions are in order, and reach for Tier 2 only when you feel the second-model blind spots you can’t otherwise close. You don’t need the full three-tier stack on day one — you need a plan that lets you get there without burning out on false positives along the way.