Access Control and Security Systems

TECHNOLOGY put to the test

June 1, 2003
By Corrina Stellitano

Employees and consumers have become accustomed to the routines of security: enter your PIN or password; smile for the camera, stare into the lens; press your finger here; insert your hand there. Personal experience tells us when biometric security precautions work — we are allowed access to our workplace, bank account or computer network without delay.

But personal experience isn't enough when it comes to selecting and purchasing biometric technologies. Users may well ask: Just how effective are today's biometric solutions?

When distinguishing between a variety of technologies and a crowd of providers, it is tempting to rely on vendors' promises of accuracy. Biometric industry experts caution, however, that the numbers alone do not add up to the whole story.

"A biometric solution must be carefully tailored to the problem at hand," says Cathy Tilton, steering committee chair for the BioAPI Consortium, an organization that works to create biometric software standards. Tilton encourages users to consider the following factors when selecting biometric tools:

• Intrusiveness/Ease of use. Can employees easily acclimatize? Will it create other holes in an access control/security system?

• Cost.

• Distinctiveness. How unique — and therefore effective — is the biometric?

• Long-term stability.

• Potential interference. Changing conditions — light or noise, for example — can challenge some biometric tools.

• Public acceptance. Acceptance rates can be greatly affected by education and training.

• Scalability.

"There are a lot of decisions to make when implementing a system, although the first thing people ask is ‘What's your accuracy?'" Tilton says. "Sweeping statements can be made about whether one technology trumps another, but it might not be true for a particular product. It's very vendor-dependent."

The success of biometric systems is also consumer-dependent. "The environment you are designing for is part of the equation and, to a certain extent, your audience is as well," says Ron Sutton, chairman of the International Committee for Information Technology Standards (INCITS) Task Group M1.4 on Biometric Performance Testing and Reporting.

When considering statistics on effectiveness, experts say the test is as important as the results. Users should always seek out third-party tests, and then ask questions. Typically, biometrics will be evaluated using a technology test (which uses stored samples to test only the algorithm), a scenario test (which attempts to mimic real-world conditions) or operational testing (conducted in the field).

Tilton says understanding the conditions under which the technology is tested is key to understanding overall test results. "How big was the pool of samples?," she asks. "What were the demographics of the people tested?"

Adds Sutton: "[Ask] what constitutes a trial, and how are the numbers computed?" Sutton is systems integrator for the FBI's Integrated Automated Fingerprint Identification System (IAFIS) at Lockheed Martin, Bethesda, Md. When designing systems that combined several biometric tools at work, Sutton says he realized how flexible the statistics could become.

"I found after testing some of these products that I couldn't trust the reported data, or interpret it properly to serve the needs of my clients," he recalls. "Defining clearly our terms is an important part of reporting useful accuracy statistics."

The BioAPI Consortium and the INCITS M1 Task Groups are also working to make sure biometric systems retain their long-term effectiveness. The Consortium's biometric software standard was recently accepted by the American National Standards Institute (ANSI).

The M1.1 Task Group on Biometric Data Interchange Formats aims to standardize how fingerprint, face, iris and signature data are stored in templates or as images, allowing the data to be used in software by any biometric provider. M1.2, the Task Group on Biometric Technical Interfaces, hopes to standardize the communication between elements — sensors, algorithms and templates — freeing purchasers from being bound to proprietary systems.

"If I buy algorithm A today from a vendor, and an alternative comes out that is revolutionary, I wouldn't want to have to throw out my existing system to take advantage of those capabilities," Sutton says.

The statistics are not entirely misleading. Several third-party tests have yielded conclusive results for specific biometric technologies. Understanding the results, however, requires a working knowledge of the terminology.

Phrases used to describe the effectiveness of a biometric system are most often its "false reject rate" (or false non-match rate), its "false accept rate" (or false match rate), and the "equal error rate." The false reject rate is the percentage of authorized entrants the system turns away. The false accept rate is the percentage of unauthorized entrants the system allows to enter, and the equal error rate is the point at which the two factors intercept. While it may not be useful in real-world applications to arrange a biometric system so the two false rates are equal, the equal error rate is sometimes helpful in comparing systems.

Important to consider also are the "failure-to-enroll" and "failure-to-acquire" rates. This is the percentage of the population that is unable to present a suitable entry sample or enroll in the biometric system. For example, it is estimated that 1-2 percent of the population will be unable to provide a workable fingerprint.

Including the failure to enroll rate is essential, says Trevor Prout, marketing director for the International Biometric Group. Each summer, International Biometric Group conducts its ‘Comparative Biometric Testing,' of 10 to 12 biometric systems across the technologies. "It doesn't matter if everyone who is able to enroll verifies correctly, if 20 percent of the population is unable to enroll," Prout says.

In various tests of one-to-one verification and one-to-many identification, fingerprint matching has been found to be more accurate than facial recognition. In tests on databases of 10,000 subjects by the National Institute for Standards and Technology (NIST), the identification accuracy of a single finger was 90 percent, while the accuracy for the face was 77 percent. For a database of 1,000 subjects, the finger ranked at 93 percent — the face at 83 percent.

The fingerprint matching used in one-to-one verification must be distinguished from the automated fingerprint identification system (AFIS) used to identify one sample among many. AFIS systems, like the one used by the FBI, conduct one-to-many identifications using multiple rolled or flat prints. Fingerprint matching is often used for verification in access control or network logons. A presented fingerprint is compared to a stored sample by two methods — minutiae or pattern comparison — and several types of sensors are available.

Fingerprint biometric systems are capable of very low levels of false acceptance and false rejection, but it is difficult to specify an accurate average error rate across the field of providers. To underscore this difficulty, consider that in the 2002 Fingerprint Verification Competition — sponsored by the University of Bologna, Michigan State University, and San Jose State University — fingerprint technology vendor Bioscrypt was found to have a 0.19 percent equal error rate. A 2001 Biometric Product Test by the Centre for Mathematics and Scientific Commuting in Middlesex, U.K, found three other fingerprint systems had equal error rates between one and 10 percent.

Tests by NIST could soon contribute more data on the effectiveness of fingerprints and facial recognition as biometric solutions. When the Patriot Act was passed in Oct. 2001, NIST was tasked with developing standards for accuracy and interoperability of biometrics for the nation's entrance/exit system. By the end of 2004, biometrics — either face, fingerprint or iris — will be used to identify new visa applicants, and will verify the identity of visa and passport holders, explains Charles Wilson, manager of the Image Group, Information Access Division, Information Technology Laboratory, NIST.

Scientists in the NIST labs have been working to test the viability of such a huge project, but this time, they are armed with larger test samples. "Before the Sept. 11 attacks, most people tested with 1,000 subjects max. Now, I have 35 million fingerprints from eight million people and six million faces from six million people," Wilson says. "We've gone from testing sample sizes in the thousands to sample sizes in the millions, and we've gone from testing material gathered in a lab to material gathered in the field."

NIST, along with the Defense Advanced Research Project Agency, the National Institute of Justice and other federal agencies, sponsored the Face Recognition Vendor Test 2002. The test used facial images from more than 37,000 individuals to test facial recognition capabilities. In 2000, the three major algorithms had shown accuracy ratings at 80 percent. In 2002, the three top facial algorithms achieved 90 percent accuracy.

The U.K. Biometric Product Test found zero percent failure to enroll when one facial recognition algorithm was tested on a much smaller sample of 200 in an office environment. The same test found an equal error rate of approximately 10 percent.

It is important to note that the images used in large-sample tests like the Vendor Test are taken in controlled conditions, with neutral gray backgrounds, shadowless lighting, and no facial expressions. "Without the controlled conditions, results could plummet as low as 47 percent," Sutton cautions.

A worldwide patent restricts production of the core technologies of iris recognition to Moorestown, N.J.-based Iridian, and no large samples exist to conduct third party testing. Its one-to-many identification capabilities have not been proven in large-scale applications, though it is "theoretically capable," Prout says.

The iris is respected as a very distinct biometric with more than 250 independent datapoint equivalents, and "it's capable of very low levels of false acceptance," Prout continues. According to Iridian data, false acceptance rates of 3.92 Χ 10-6 are achievable for verification applications. The U.K. test found the Iridian iris system had no false matches in more than two million cross comparisons of 200 subjects.

Failure to enroll rates for the iris recognition should be evaluated when selecting a system. "People get used to using it," Prout says. "But the first time they interact with one of these machines, it is not necessarily natural." Most systems notify the user of improper iris positioning.

Heralded as the most commonly used biometric system for time and attendance or access control, hand geometry has been in use for more than a decade. The hand geometry system measures the sizes and depths of the fingers and hand. Templates which absorb subtle changes can help dispel the effects of weight gain or loss. False rejection rates could increase if the user only checks into the system infrequently — every six months, for example.

According to the U.K. Biometric Product Test, one hand geometry system displayed equal error rates of slightly more than one percent. The sample group tested in the U.K. had no failures to enroll. Hand geometry vendor Recognition Systems, Campbell, Calif., cites tests by the Department of Energy's Sandia National Labs and the United Kingdom's National Physical Laboratory which found its system to have equal error rates of 0.1 and 0.4 percent, respectively.

Several other technologies continue to progress toward widespread use. Voice authentication, distributed by companies like Menlo Park, Calif.-based Nuance, Boston-based Speechworks, and Ottawa-based OTG, works to add security in situations where the user is already communicating by voice. For example, it would work well to verify the identity of a banking client who traditionally voices his social security number for account access.

According to a May 2000 report by The Centre for Communication Interface Research at The University of Edinburgh, Nuance's algorithm offers an equal error rate of 0.9 percent, or 99.1 percent accuracy.

Signature scan technology, produced by Redwood Shores, Calif.-based CIC, as well as vendors in Israel and Japan, evaluates and verifies users according to how they sign their names. Designed for one-to-one verification, the technology is commonly used for access security on mobile devices which have touch-screens, and for work flow automation.

While CIC's system touts a zero false accept rate, user errors can sometimes contribute to a higher false reject rate. Overall, CIC officials say their system can achieve an equal error rate of 0.17 percent.

The patented keystroke dynamic technology is offered only by Bellevue, Wash.-based BioNet Systems and is in use at approximately 1,000 computers across the nation. The software measures "flight and dwell time," or how much time between key pressings and how long the key is pressed. This technology is well-suited for providing incremental security protection when entering a password or keycode.

Stanford research in the 1980s found accuracy ratings of 98.4 percent for the technology, if users entered the eight-character ID the requisite 15 times to enroll correctly. Though Gordon Ross, BioNet's chief security and technology officer, says there is a zero failure to enroll rate, wireless keyboards may necessitate a slightly lower security setting.