Comparative Biometric Testing

>   Round 6 Public Report available for download

IBG's Comparative Biometric Testing (CBT) is the industry's longest-running scenario test effort. CBT measures the accuracy and usability of commercial fingerprint, face recognition, iris recognition, voice recognition, signature recognition, hand geometry, and emerging biometrics systems.

>   Participant Overview (pdf)

CBT Benefits

Broad Dissemination of Results to Decision-Makers across Government and Industry. The CBT Public Report, including performance data for all evaluated systems, is relied upon by IBG’s worldwide audience of thousands of decision-makers and industry professionals in hundreds of government and commercial organizations.

Access to Detailed System Performance Data. Reliable performance data relative to competing modalities and systems is difficult to collect. CBT generates benchmark performance data essential to engineering teams, product developers, and sales and marketing staff. In addition to the core test metrics, CBT generates demographic- and test subject-specific performance data, helping vendors identify areas for improvement in their capture and matching technologies.

Independent, Third-Party Biometric Performance Certification.Biometric system accuracy and usability are essential considerations for government and commercial deployers, technology partners, and investors. Third-party testing by credible and experienced testing organizations is a precondition of procurement for many deployers. IBG has been recognized as the industry's leading source of objective performance data since the late 1990's.

Comparative Biometric Testing addresses the following questions critical to successful deployments:

• How do leading technologies compare in terms of False Match Rates, False Non-Match Rates, and Failure to Enroll Rates?
• How well do technologies perform under real-world conditions with typical users?
• Which technologies are able to enroll the largest percentage of users?
• What percentage of enrolled users are falsely matched and non-matched? 
• Do error rates increase over time?
• How well do biometric devices address ergonomic and ease-of-use issues? 
• Which technologies and devices are ready to deploy in operational environments?

Understanding biometric technologies' accuracy and performance under real-world conditions is a precondition of effectively deploying biometrics. Error rates encountered by actual users - including false match rates, false non-match rates, and failure to enroll rates - often differ from error rates generated in laboratory tests using databases of biometric samples.

Standards Compliance

CBT conforms to the following approved U.S. standards for biometric performance testing.

>   ANSI INCITS 409.1-2005
     Biometric Performance Testing and Reporting - Part 1: Principles and Framework

>   ANSI INCITS 409.3-2005
     Biometric Performance Testing and Reporting - Part 3: Scenario Testing and Reporting

The CBT Test Director, Michael Thieme, is the editor of ANSI INCITS 409.3-2005 and ISO/IEC JTC1 SC37 19795-2: Testing Methodologies for Technology and Scenario Evaluation. 

CBT sponsors have included Honeywell, Microsoft, American Airlines, Lockheed Martin, EDS, Fidelity Investments, Star Systems, and the Financial Services Technology Consortium.

CBT Downloads

>   Test Plan (pdf)
>   Requirements (pdf)
>   Biometric Performance Certification FAQ (pdf) 
>   IBG's presentation on Lessons Learned from Comparative Biometric Testing (pdf)
>   A partial list of IBG performance report recipients (pdf)

Comparative Biometric Testing and Performance Certification

Systems tested in CBT that meet accuracy and usability criteria obtain Biometric Performance Certification. For more information on Biometric Performance Certification, please contact Michael Thieme at International Biometric Group.

CBT History

IBG tests between ten and twelve leading biometric systems in each Comparative Biometric Test round. To date, more than 50 biometric systems have been tested. Technologies tested to date include fingerprint, face recognition, iris recognition, voice recognition, signature recognition, hand geometry, and keystroke biometrics. Below is a partial list of systems tested in CBT.