Stop Trusting Identity Verification Accuracy Claims. Start Testing Them

Every identity verification vendor will tell you they’re accurate. Most cite a figure above 99%. Some go further: “industry-leading,” “AI-powered accuracy that outperforms human review,” “the most accurate system available.” The claims are everywhere, and they sound the same.

In February 2026, the U.S. Department of Homeland Security published results from its Remote Identity Validation Rally (RIVR), one of the most rigorous independent benchmarks ever run on commercial IDV systems. The findings were blunt: 6 out of 7 identity document validation systems tested fell short of the performance threshold for fraud acceptance, false rejection, or both. Only one system met the security target.

If your vendor selection relied on accuracy figures from a sales presentation, the RIVR results are worth sitting with.

What does “99% accuracy” actually mean?

Accuracy in identity verification is not a single number. It is a function of the conditions under which a system was tested, and most vendors do not disclose those conditions.

The variables vendors quietly omit

When a vendor quotes 99% accuracy, the first question is: 99% on what? A model trained primarily on US driver’s licences and EU passports returns very different results when it encounters a Thai national ID, an Indonesian KTP, or an Emirati residence permit. Document type mix is the single most influential variable in any identity verification accuracy benchmark, and it is rarely disclosed upfront.

Other variables that shift accuracy figures significantly: lighting conditions during image capture, camera hardware (a flagship smartphone versus a mid-range device), whether the test set included fraudulent documents or only genuine ones, and the demographic spread of the test population. A number stripped of these parameters is not a number you can act on.

Product validation accuracy vs. operational accuracy and why they diverge

There is a meaningful distinction between two accuracy measures that vendors report, often without flagging the difference. Product validation accuracy reflects how a model performs in controlled, balanced conditions, the dataset the vendor used to build and test their system. Operational accuracy is how the same system performs on your real users, with your document mix, in your target markets.

The gap between the two is predictable and, in high-diversity markets, substantial. A system achieving 99% in a controlled lab environment can fall significantly when processing a high volume of non-Latin documents or images captured in adverse lighting without a single line of code changing. The vendor’s marketing copy stays the same. Your conversion rate does not.

What independent testing actually found

Independent benchmarking closes the gap between what vendors claim and what their systems deliver. The DHS RIVR program, run by the Department of Homeland Security’s Science and Technology Directorate in partnership with the National Institute of Standards and Technology (NIST) and the Maryland Test Facility, is the most authoritative public benchmark currently available for commercial IDV technology.

The document validation gap

The Track 2 results, published in February 2026, evaluated 7 anonymised identity document validation systems against genuine and fraudulent documents from Maryland and California, with fraudulent samples supplied by the DHS Homeland Security Investigations forensic laboratory. The performance distribution was stark.

6 of 7 systems fell short on fraud acceptance rate (FAR), false rejection rate (FRR), or both. One system posted a false rejection rate above 97%, blocking nearly every document it saw — a number that reads as secure until you realise it signals a system that cannot function as an onboarding tool. Another posted a false acceptance rate above 13%, passing a troubling proportion of fraudulent documents as genuine. Only one of the seven met the security threshold as measured by false acceptance rate.

These are not prototype systems. These are commercial identity document validation technologies that, before February 2026, were presumably telling prospective buyers they were accurate.

The selfie-to-ID gap

Track 1 of RIVR evaluated 16 systems on selfie-to-document matching, the biometric step that confirms the person presenting the ID is the person in the photo. Results here were more encouraging but still pointed: only 5 of 16 systems met all of DHS’s performance goals. That is a pass rate of 31% among commercial vendors, for one of the most foundational tasks an IDV system performs.

The takeaway is not that identity verification cannot be done accurately. It is that accuracy at the level vendors claim, across the conditions buyers need, is rarer than the market implies.

What a trustworthy accuracy claim looks like

A trustworthy accuracy claim is specific, independently verified, and attached to a disclosed methodology. That narrows the field considerably.

The standard that now separates real from marketed

For biometric liveness and presentation attack detection, the benchmark that matters is ISO/IEC 30107-3, the international standard for testing whether a face verification system can distinguish a live person from a spoof. iBeta Quality Assurance, accredited by NIST’s National Voluntary Laboratory Accreditation Program (NVLAP), administers conformance testing against this standard.

In June 2025, iBeta introduced Level 3 conformance specifically in response to the rise of AI-generated attacks, including deepfakes and face-swap techniques. Level 3 tests expert attackers operating with no budget constraints and weeks to attempt a breach of the conditions that most closely replicate a motivated adversary. Very few vendors globally hold Level 3 conformance under this standard, and Shufti holds a Level 3 iBeta certificate is one of them.

For document validation, DHS RIVR is the most rigorous public reference currently available. A vendor that has participated in RIVR and can disclose their system’s results or identify themselves among the Track 1 systems that met all performance goals is demonstrating a standard of transparency that vendor-commissioned testing cannot replicate.

Four questions to ask before signing a contract

Before accepting any accuracy figure, press the vendor on four points.

What was the document mix in the test set?

If it skewed toward US and EU documents, the number does not represent a globally diverse user base.

Was testing conducted by an accredited independent third party?

Internal benchmarks have structural conflicts of interest; named, accredited labs are the reference points that matter.

What attack classes were included?

A liveness system tested only against printed photos and basic video replay has not been evaluated against the attacks that are relevant today.

Can you share the test methodology, not just the headline figure?

A vendor who can answer that question with specificity is a vendor whose accuracy claim is worth considering.

How Shufti approaches accuracy

The accuracy gap hits hardest in the markets that matter most. Most vendors weren’t built for Vietnam, Indonesia, Brazil, South Asia, or the Gulf. They trained on Western IDs and bolted on everything else, which means their lab figures reflect conditions that do not map to those documents in the field.

Shufti’s document verification was trained natively on 10,000+ document types across 240+ countries and jurisdictions. The model was built from the ground up to handle that range, not patched to accommodate it after the fact. On the biometric side, Shufti holds iBeta Level 3 conformance under ISO/IEC 30107-3, the standard introduced in June 2025 for AI-driven attack defence. Expert attackers, no budget constraints, weeks of attempts. That is the bar the conformance was earned against, and it is a claim you can test.

See how Shufti’s identity verification holds up on your actual document mix. Book A Demo

Blog banner image prompt

An abstract editorial composition for identity verification accuracy. A single identity document lies on a cold, clinical surface under a focused examination light, casting a long directional shadow to the right. The document is featureless — no text, no logos, no real ID content. The scene renders in deep navy (#0D1428) with a single red (#F24348) light source striking the document edge. One thin ray of sky blue (#539FF9) catches the surface texture to the left. The mood is forensic and restrained — a document under scrutiny, not celebration. Wide 16:9 composition; document sits left of center with shadow extending rightward into negative space. Cinematic depth of field, editorial lighting style, modern and confident.

No embedded text, no typography, no letters, no real people, no identifiable faces. No competitor brand cues. No literal product UI or dashboards. No stock-photo cliches (handshake, padlock-on-screen, hooded hacker, fingerprint-on-circuit, faceless suit, magnifying glass).