Officially corrected errors persist in AI training data
Officially corrected errors persist in AI training data
Céphéide, Loxyn's measurement bench, published a public alert with data, scripts and proofs examining whether official corrections—EU corrigenda, verified RFC errata and retracted papers—are present in public datasets used to train language models (EurlexResources, FineWeb-2, a FineWeb sample, peS2o). It reports that 95.5% of testable single-language corrections to EU acts (1,989/2,082) remain in erroneous form in EurlexResources, that about 12,200 retracted papers are in peS2o with 79% lacking any mention of the retraction, and that five models asked without documents give the erroneous version in 45% of answers where they commit to one version. Each step was preregistered and timestamped with OpenTimestamps in the Bitcoin blockchain, and failed controls and deviations are reported.
Source: Hacker News AI · zenodo.org