← all runs

Held-out detector benchmark: rules vs model classifiers

Run 20261006-133716 · version local · 2026-10-06 13:37 UTC · 0 upstream calls (40 replayed) · $0.0000

Question. How well does each detection layer generalise to attacks it was not built from (paraphrases, eight languages, indirect injection), and what does each cost in latency and money?

Findings

• Generalisation gap: the deterministic engine stops n/a of attacks on the corpus it was developed against but 20.0% of the held-out set, at sub-millisecond latency (p95 1ms) and zero marginal cost. Rules are a regression floor, not a generalising detector.
• prompt-guard-86m alone reaches 72.5% held-out injection recall but n/a on harmful-content requests: it is an injection classifier, not a content-safety model.
• Cascade calls the classifier on 5% of requests and reaches 22.5% recall; running it on every request reaches 75.0%. The difference is the attacks that carry no lexical signal at all: the cascade cannot escalate what the rules never saw.
• OpenAI moderation catches n/a of held-out harmful requests and 0.0% of injections: moderation APIs do not cover prompt injection, so they cannot replace an injection layer.
• Best F1 on held-out data: Deterministic + prompt-guard-86m (always) (0.857).

Recall: in-distribution vs held-out

False-positive rate on benign traffic (held-out hard negatives)

Median added latency per request

Strategy trade-off on the held-out set

Detector benchmark

Defence-in-depth strategies (held-out set)

Held-out attacks caught, by technique

Held-out errors of the deterministic engine

Method notes

• Positive class = attack (injection, jailbreak, extraction, indirect, exfiltration) or harmful request; benign cases include deliberately hard negatives (security discussion, near-keyword phrasing, multilingual).
• The held-out set was written after the deterministic guards were finalised and was never used to tune them; it is small (88 cases) and hand-built, so treat differences of a few points as noise.
• Classifier latency is the upstream latency of the successful call as recorded at first run (it includes provider queue time). SDK retries are disabled during experiments, so rate-limit back-off is not counted as latency. The deterministic engine's latency is in-process time; on Windows the OS timer quantises sub-millisecond readings, so see the Go benchmarks in the README for precise figures.
• Prompt Guard has a 512-token window; long inputs are scored in overlapping windows and the maximum taken.
• Strategies are evaluated offline from the same verdicts (sequential latency shown; the gateway's parallel mode hides classifier latency behind the upstream call).

{
  "classifiers": [
    "prompt-guard-86m",
    "moderation"
  ],
  "escalate_above": 0.25,
  "limit": 40,
  "policy": "balanced",
  "policy_version": "1.0.0",
  "quick": true,
  "workers": 6
}