Community Trust ScoreVerified
Stronger AI models repeated forged answers when weaker helper agents read the evidence, according to a study submitted to arXiv on Nov. 12. The research, conducted by Keegan Wang and Anantika Mannby, tested agents on fictional records and two-option questions to measure reliability risks in multi-model systems.
More coverage is filed under Technology.
The study defined “harmful revisions” as the adoption of an attacker’s wrong answer against a correct baseline. The findings indicate that while models can resist forged data when reading files directly, their accuracy drops sharply when they delegate the reading task to a less capable sub-model.
Delegation increases error rates
In tests using Anthropic’s Claude Code, an Opus 5 supervisor chose a planted wrong answer in only 1 of 180 trials when it read the files itself. However, when the Opus 5 supervisor delegated the reading task to Haiku 4.5, it chose the planted wrong answer in 75 of 180 trials. In that delegated configuration, the supervisor was mechanically limited to its Task tool, which removed direct file access.
A separate test using OpenAI’s Codex showed a similar pattern. GPT-5.6-sol made no harmful revisions in 180 direct-reading trials. But when GPT-5.6-sol relied on GPT-5.6-terra to read the evidence, it made 44 harmful revisions in 180 trials. The Codex delegation restriction was instructional and checked per trial, differing from the mechanical limit applied in the Claude Code tests. The researchers noted that the Claude Code and Codex comparisons used different attacks, which limits direct comparisons between the two products.
Attack methods and mitigations
The study included a “stacked-forgery” condition that combined several forms of false evidence. This attack placed five forged documents in a 35-file store. The forged documents included correction notices, an authoritative annotation, and repeated false accounts. The detectable forgeries in the study predated the records they purported to correct and lacked genuine accession lines.
In the Codex test, a forged annotation claiming authority was used. A correction-notice condition fooled Codex in 172 of 180 direct-reading trials. The researchers found that instructions directed at automated readers were resisted wherever they were tested. However, some successful attacks looked like factual records, allowing a helper’s report to carry a false conclusion upward to the supervisor.
Using same-tier helpers reduced failures in the tested comparisons. Claude Code recorded no harmful revisions in 180 trials per condition under the annotation and stacked attacks when using same-tier helpers. Codex recorded 1 harmful revision in 180 trials under the annotation attack when using same-tier helpers. Clean-store delegation produced no harmful revisions in 180 trials for each product.
Testing in real-world harnesses
In a real-harness test, requiring original-record quotations reduced harmful revisions from 15 to 3. A verifier told to be skeptical reduced harmful revisions to 1. Each of these conditions involved 40 trials per arm. The authors identified lost provenance and reduced checking as possible contributors to the failures. Their evidence about fewer tool turns is correlational, leaving suppressed verification as a hypothesis.
The experiments measured performance on generated fictional tasks in a sandbox. The study offers no estimate of ordinary customer failures.
