Coinbase’s latest test of artificial-intelligence models found that newer versions of three major model families detected fewer fraudulent payments and a smaller share of fraud value than their predecessors. The comparison used the same decision policy throughout, meaning the models were tested within an unchanged payment-screening setup rather than against redesigned systems.
The result challenges a common assumption in fraud prevention: that replacing an existing model with a newer version will automatically improve performance. According to CryptoSlate, Coinbase reported the findings on Oct. 7 after replaying historical payment activity from its Onramp service.
Newer models performed worse in a fixed replay
The historical evaluation covered 16,140 transactions involving 7,293 users. It included 813 confirmed fraudulent transactions and drew on a nine-week period before Coinbase introduced its risk agent. Fraud cases that had matured were retained, while legitimate transactions were sampled for the comparison.
Each candidate model assessed recent transaction behaviour using the same guidance. Coinbase also kept the policy that converted risk classifications into payment decisions unchanged. This design was intended to isolate how the model versions behaved under the existing configuration, rather than measure the effect of a broader system overhaul.
Coinbase compared Opus 4.5 with Opus 5, Sonnet 4.6 with Sonnet 5, and GPT-5.4 with GPT-5.6 (sol). All three newer versions recorded lower recall, lower F1 scores and lower dollar-weighted recall.
Recall measures the proportion of fraudulent cases identified by a model. Dollar-weighted recall measures the proportion of the total fraudulent value detected, making it possible to distinguish between catching many low-value cases and catching a larger share of the money involved. F1 combines precision and recall into a single measure.
Sonnet’s recall dropped by 22.2 percentage points, while its dollar-weighted recall declined by 22.9 points. Opus recorded a smaller 0.8-point fall in recall, but its newer version also had lower precision. In practical terms, a lower precision result means that a smaller share of transactions flagged as fraudulent were actually fraudulent.
GPT’s results illustrated why focusing on one metric can give an incomplete picture. Precision improved by 11.5 percentage points, but recall fell 20.7 points and dollar-weighted recall dropped 21.8 points. The newer GPT version produced more accurate fraud flags among the transactions it selected, while allowing more fraudulent cases and value to go undetected in the replay.
Coinbase said the test did not establish that customers suffered losses because of the model upgrades. It also said the evaluation identified performance regressions without determining their cause.
A custom model produced a different result
Coinbase’s findings do not mean that every model change worsens fraud detection. In a separate disclosure on Oct. 8, the company said a post-trained Qwen3.5-9B model outperformed Opus 4.5 on four fraud-detection measures.
The custom model’s F1 score was 9.6 percentage points higher, while its dollar-weighted recall increased by 35.4 points. Coinbase specialised the model with historical fraud outcomes and deterministic rewards designed to balance fraudulent and legitimate examples.
The company also reported a separate production-latency comparison. Median end-to-end latency for an LLM request was 0.683 seconds for the custom model, compared with 1.515 seconds for Opus 4.5, a relative reduction of 55%. The latency measurements and the fraud-detection benchmark came from separate evaluations, so the figures do not represent one combined test.
Coinbase had previously tested adding selective LLM review to its existing models and rules. That agent-enabled flow recorded 30% fewer fraudulent transactions and 22% less fraud value, but it was not a comparison between newer and older versions of the same model families.
Why the testing approach matters
For payment providers, the findings underline the importance of testing a model within the precise policy and workflow where it will operate. A model can improve one screening statistic while weakening overall fraud coverage, particularly if its precision rises as recall falls.
Coinbase recommends first testing the full configuration under consideration, then assessing changes to prompts or thresholds separately. It also identifies latency, reliability and cost as factors to evaluate alongside detection quality.
The research has limitations. The underlying proprietary dataset cannot be released, according to the researchers, which limits independent replication and the ability to determine how broadly the results apply. The historical replay also measured model behaviour in the tested setup; it did not quantify customer losses caused by deploying the newer versions.
Market picture
Moneyiar market data recorded at the stated times shows a mixed but generally weaker picture among the listed crypto assets:
- Cronos (CRO): $0.0594276794, down 2.39% over 24 hours and 13.15% over seven days, as of Oct. 11, 2026, at 11:54 +03:30.
- Decimal (DEL): $0.3992247672, up 3.01% over 24 hours and down 1.38% over seven days, as of 11:53 +03:30.
- AKEDO (AKE): $0.0261224759, down 4.36% over 24 hours and 26.3% over seven days, as of 11:54 +03:30.
- BAS (BAS): $0.0219436393, down 1.15% over 24 hours and 8.7% over seven days, as of 11:53 +03:30.
These figures are Moneyiar data and describe market prices and percentage changes, not the performance of Coinbase’s fraud-detection models.
