Pangram’s Spero finds AI detection fails against GPT-4

Max Spero, the chief AI officer at Pangram, warned that detecting AI‑generated text is becoming harder than telling whether a story is real or fake, saying the task is “a moving target” in a tweet on Thursday. He said Pangram’s internal research shows that the accuracy of existing detection tools drops by more than 20 % when applied to the latest large language models, and that the gap is widening each month. Spero added that the industry must rethink its approach to verification if it wants to keep up with the pace of AI development.

The problem is rooted in the rapid evolution of language models. As models grow larger and incorporate more diverse training data, they produce text that mimics human style with increasing fidelity. Detection systems that rely on statistical fingerprints or token‑frequency patterns are struggling to keep up, because the differences between human and machine output are shrinking. Spero explained that earlier models left obvious tell‑tale signals—such as repetitive phrasing or unnatural syntax—that detectors could flag. Newer models, however, generate more varied and context‑aware prose, blurring the line between human and AI authorship. The result is a “cat and mouse” game in which detection tools lag behind every major release.

Pangram’s research team has tested over a dozen detection frameworks against a benchmark set of texts generated by GPT‑4, Claude‑2, and Gemini‑1. The results were sobering: only 48 % of samples were correctly identified as AI‑generated by the best‑performing tool, and accuracy fell to 35 % for the most recent models. Spero highlighted that this decline is not just a technical issue but a societal one, as misinformation campaigns increasingly use sophisticated AI to create convincing but false narratives. He urged media outlets to adopt multi‑layered verification, combining automated detection with human editorial review and source‑tracing techniques.

Looking ahead, Pangram is investing in a new detection architecture that uses multimodal cues—such as metadata, usage patterns, and cross‑model comparisons—to improve reliability. Spero said the company plans to release a beta API next quarter that will allow publishers to flag suspect content in real time. He also called on regulators to consider standards for AI‑generated content, arguing that without clear guidelines, the risk of deep‑fake text will only grow. The conversation around AI detection is gaining urgency as more platforms and governments grapple with the ethical and practical implications of machine‑written prose.