Facial recognition – fascinating and intriguing
biometric technologies are sparking the imagination quite like
Equally, its arrival has prompted profound concerns and surprising reactions in 2019. But more about that later.
artificial intelligence (AI) and the
blockchain, face recognition certainly represents a significant digital challenge for all companies and organizations - and especially governments.
In this dossier, you'll discover the 7 face recognition facts and trends that are set to shape the landscape in 2019-2020.
- Top technologies and providers
- AI impact - Getting better all the time
- Market dynamics and dominant use-cases
- Mapping of new users
- The legal system - Privacy vs Security: and the winner is...
- Face recognition latest hacks
- Towards hybridized solutions.
Let’s jump right in.
How face recognition works
Facial recognition is the process of identifying or verifying the identity of a person using their face. It captures, analyzes and compares patterns based on the person's facial details.
- The face detection process is and essential step as it detects and locates human faces in images and videos.
- The face capture process transforms an analog information (a face) into a set of digital information (data) based on the person's facial features.
- The face match process verifies if two faces belong to the same person.
Today it's considered to be the most natural of all biometric measurements.
And for good reason – we recognize ourselves not by looking at our fingerprints or irises, for example, but by looking at our faces.
Gemalto, a world leader in digital security, has specialized in the sensitive field of biometric technologies for almost 30 years. The company has always collaborated with the best technical and institutional players in the sector when it comes to research, ethics and biometric applications.
Before we go any further, let's quickly define "identification" and "authentication".
Face recognition data to identify and verify
Biometrics are used to identify and authenticate a person using a set of recognizable and verifiable data unique and specific to that person.
For more on
biometrics definition visit our web dossier on biometrics.
Identification answers the question: "Who are you?"
Authentication answers the question: "Are you really who you say you are?".
Stay with us, here are some examples :
- In the case of facial biometrics, a 2D or 3D sensor "captures" a face. It then transforms it into digital data by applying an algorithm, before comparing the image captured to those held in a database.
- These automated systems can be used to identify or check the identity of individuals in just a few seconds based on their facial features: spacing of the eyes, bridge of the nose, contour of the lips, ears, chin, etc.
They can even do this in the middle of a crowd and within dynamic and unstable environments. Proof of this can be seen in the performance achieved by Gemalto's Live Face Identification System (LFIS), an advanced solution resulting from our long-standing expertise in biometrics.
- Owners of the
iPhone X have already been introduced to facial recognition technology. However, the Face ID biometric solution developed by Apple was heavily criticized in China in late 2017 because of its inability to differentiate between certain Chinese faces.
Of course, other signatures via the human body also exist: fingerprints, iris scans, voice recognition, digitization of veins in the palm of the hand and behavioral measurements.
Why facial recognition then?
Facial biometrics continues to be the preferred biometric benchmark. That's because it's easy to deploy and implement. There is no physical interaction required by the end user. Moreover, face detection and face match processes for verification/identification are very fast.
So, what is the best face recognition software?
#1 Top facial recognition technologies
In the race for biometric innovation, several projects are vying for the top spot.
Google, Apple, Facebook, Amazon and Microsoft (GAFAM) are also very much in the mix. All the software web giants now regularly publish their theoretical discoveries in the fields of artificial intelligence, image recognition and face analysis in an attempt to further our understanding as rapidly as possible.
The very latest results of tests conducted in March 2018 and published in May by the US Homeland Security Science and Technology Directorate, known as the
Biometric Technology Rally, also provide an excellent indication of the best face recognition software available on the market.
But let’s take a closer look :
GaussianFace algorithm developed in 2014 by researchers at Hong Kong University achieved facial identification scores of 98.52% compared with the 97.53% achieved by humans. An excellent score, despite weaknesses regarding memory capacity required and calculation times.
Facebook and Google
Again in 2014, Facebook announced the launch of its
DeepFace program which can determine whether two photographed faces belong to the same person, with an accuracy rate of 97.25%. When taking the same test, humans answer correctly in 97.53% of cases, or just 0.28% better than the Facebook program.
In June 2015, Google went one better with
FaceNet, a new recognition system with unrivaled scores: 100% accuracy in the reference test
Labeled Faces in The Wild, and 95% on the YouTube Faces DB. Using an artificial neural network and a new algorithm, the company from Mountain View has managed to link a face to its owner with almost perfect results.
This technology is incorporated into
Google Photos and used to sort pictures and automatically tag them based on the people recognized. Proving its importance in the biometrics landscape, it was quickly followed by the online release of an unofficial open-source version known as
Microsoft, IBM and Megvii
A study done by MIT researchers in February 2018 found that Microsoft, IBM and China-based Megvii (FACE++) tools had high error rates when identifying darker-skin women compared to lighter-skin men.
At the end of June, Microsoft announced in a blog post that it had made solid improvements to its biased facial recognition technology.
In May 2018,
Ars Technica reported that Amazon is already actively promoting its cloud-based face recognition service named Rekognition to law enforcement agencies. The solution could recognize as many as 100 people in a single image and can perform face match against databases containing tens of millions of faces.
Newsweek reported that Amazon’s facial recognition technology falsely identified 28 members of US Congress as people arrested for crimes.
Key biometric matching technology providers
At the end of May 2018, the US Homeland Security Science and Technology Directorate published the results of sponsored tests at the Maryland Test Facility (MdTF) done in March. These real-life tests measured the performance of
12 face recognition systems in a corridor measuring 2 m by 2.5 m.
Gemalto's solution utilizing a
Facial recognition software (LFIS) achieved excellent results with a face acquisition rate of 99.44% in less than 5 seconds (against an average of 68%), a Vendor True Identification Rate of 98% in less than 5 seconds compared with an average 66%, and an error rate of 1% compared with an average 32%.
March 2018 – The
live testing done using more than 300 volunteers identified the best-performing facial recognition technologies.
Facial emotion detection and recognition
Emotion recognition (from real-time of static images) is the process of mapping facial expressions to
identify emotions such as disgust, joy, anger, surprise, fear or sadness on a human face with image processing software.
Its popularity comes from the
vast areas of potential applications.
It's different from facial recognition which goal is to identify a person not an emotion.
Face expression may be represented by geometric or appearance features, parameters extracted from transformed images such as
eigenfaces, dynamic models and 3D models.
Providers include Kairos (face and emotion recognition for brand marketing), Noldus, Affectiva, Sightcorp, Nviso among others.
#2 Learning to learn through deep learning
The feature common to all these disruptive technologies is known as Artificial Intelligence (AI) and more precisely deep learning where a system is capable of learning from data.
Why is it important?
It's a central component of the latest-generation algorithms developed by Gemalto and other key players in the market, and holds the secret to face detection, face tracking and face match as well as real-time translation of conversations.
Face recognition systems are getting better all the time.
According to a recent NIST report, massive gains in accuracy have been made in the last 5 years (2013-
2018) and exceed improvements achieved in the 2010-2013 period.
Most of face recognition algorithms in 2018 outperform the most accurate algorithm from late 2013.
Artificial neural networks are algorithms supplied with several different input values. These are processed by a range of functions which eventually return one output value. These functions initially involve a learning phase in order to calibrate the results produced.
- Firstly, the network is supplied with input values and known output results.
- Checks are then made to ensure that the network is producing the expected result.
- As long as this is not the case, adjustments are made until the system is correctly configured and capable of systematically producing the expected result.
Think about it this way :
The network behaves in a similar way to a black box. It is given input values whose results are not yet known, and will produce an output value.
So artificial neural network algorithms are helping face recognition algorithms to be more accurate.
Find out more in the video below.
#3 Facial recognition markets
Face recognition markets
A study published in June 2019, estimates that by 2024, the global face recognition market would generate $7 billion of revenue, supported by a compound annual growth rate (CAGR) of 16% over the period 2019-2024.
For 2019, the market is estimated at $3.2 billion.
The two biggest drivers of this growth are surveillance in the public sector and numerous other applications in diverse market segments.
According to the study, the top facial recognition vendors include Accenture, Aware, BioID, Certibio, Fujitsu, Fulcrum Biometrics, Gemalto, HYPR, Idemia, Leidos, M2SYS, NEC, Nuance, Phonexia, Smilepass, Fulcrum Biometrics.
The main facial recognition applications can be grouped into three key categories.
Top 3 application categories
1. Security - law enforcement
This market is led by increased activity to combat crime and terrorism.
The benefits of facial recognition systems for policing are evident: detection and prevention of crime.
- Facial recognition is used when issuing identity documents, and most often combined with other biometric technologies such as fingerprints.
- Face match is used at border checks to compare the portrait on a digitized biometric passport with the holder's face.
- In 2017, Gemalto was responsible for supplying the new automated control gates for the
PARAFE system (Automated Fast Track Crossing at External Borders) at Roissy Charles de Gaulle airport in Paris. This solution has been devised to facilitate evolution from fingerprint recognition to
facial recognition during the course of 2018.
Face biometrics can also be employed in police checks although its use is rigorously controlled in Europe. In 2016, the "man in the hat" responsible for the Brussels terror attacks was identified thanks to FBI facial recognition software. The South Wales Police implemented it at the UEFA Champions League Final in 2017.
- In the United States, 26 states (and probably as many as 30) allow law enforcement to run searches against their databases of driver’s license and ID photos. The FBI has access to driver’s license photos of 18 states.
- Drones combined with aerial cameras offer an interesting combination for facial recognition applied to large areas during mass events for example. According to the Keesing Journal of documents and Identity of June 2018 (issue 56), some
hovering drone systems can carry a 10-kilo camera lens which can identify a suspect from 800 meters from a height of 100 meters. As the drone can be connected to the ground via a power cable, it has unlimited power supply. In addition, the communication to ground control can’t be intercepted as it also uses a cable.
Significant advances have been made in this area.
Thanks to deep learning and face analysis, it is already possible to:
- track a patient's use of medication more accurately
- detect genetic diseases such as
DiGeorge syndrome with a success rate of 96.6%
- support pain management procedures.
3. Marketing and retail
This area is certainly the one where use of facial recognition was least expected. And yet quite possibly it promises the most. Know Your Customer (KYC) is sure to be a hot topic in 2019. This important trend is being combined with the latest marketing advances in customer experience.
By placing cameras in retail outlets, it is now possible to analyze the behavior of shoppers and improve the customer purchase process.
Like the system recently designed by
Facebook, sales staff are provided with customer information taken from their social media profiles to produce expertly customized responses.
The American department store
Saks Fifth Avenue is already using such a system.
Amazon GO stores are reportedly using it.
How long before the selfie payment?
Since 2017, KFC, the American king of fried chicken, and Chinese retail and tech giant Alibaba, have been testing a face recognition payment solution in Hangzhou, China.
#4 Mapping of new users
While the United States currently offers the largest market for face recognition opportunities, the
Asia-Pacific region is seeing the fastest growth in the sector. China and India lead the field.
Face recognition in China
Face recognition is the new hot tech topic in China from banks and airports to police.
Now authorities are expanding the facial recognition
sunglasses program as police are beginning to use them in the outskirts of Beijing.
China is also setting up and perfecting a video surveillance network countrywide.
176 million surveillance cameras were in use at the end of 2017 and 626 million are expected by 2020.
Chinese police is working with artificial intelligence companies such as Yitu, Megvii, SenseTime and CloudWalk according to
The New York Times of 14 April 2019.
Facial recognition in Asia
Facial recognition wil be a major topic for the
2020 Olympic Games in Tokyo (Japan). This technology will be used to identify authorized persons and grant them access automatically enhancing their experience and safety.
In Sydney, face recognition is
undergoing trials at airports to help move people through security much faster and in a safer way.
In India, the Aadhaar project is the largest biometric database in the world. It already provides a unique digital identity number to 1.2 billion residents. UIDAI, the authority in charge, announced that
facial authentication will be launched by
September 15, 2018. Face authentication will be available as an add-on service in fusion mode along with one more authentication factor like fingerprint, Iris or OTP.
Other large projects
In Brazil, the Superior Electoral Court (Tribunal Superior Eleitoral) is involved in a nationwide biometric data collection project. The aim is to create a biometric database and unique ID card by 2020, recording the information of 140 million citizens.
In Africa, Gabon, Cameroon and
Burkina Faso have chosen Gemalto to meet the challenges of biometric identity.
Russia's Central Bank has been deploying a country-wide program since 2017 designed to collect faces, voices, iris scans and fingerprints. But the process is progressing very slowly according to the
Biometricupdate website of 13 March 2019.
#5 When face recognition strengthens the legal system
The ethical and societal challenge posed by data protection is radically affected by the use of facial recognition technologies.
Do these technological feats, worthy of science-fiction novels, genuinely threaten our freedom?
And with it our anonymity?
EU and UK biometric data protection
In Europe and the UK, the
General Data Protection Regulation (GDPR) provides a rigorous framework for these practices. Any investigations into a citizen's private life or business travel habits are out of the question and any such invasions of privacy carry severe penalties.
Applicable from May last year, the GDPR supports the principle of a harmonized European framework, in particular protecting the right to be forgotten and the giving of consent through clear affirmative action. This directive is bound to have international repercussions.
Yes, you read it well. There's now one law for 500 million people.
U.S. biometric data protection landscape
In America, the State of Washington was the third US state (after Illlinois and Texas) to formally protect biometric data through a new law introduced in June 2017.
The California Consumer Privacy Act (CCPA) passed in June 2018 and effective as of 1 January 2020 will have a serious impact for privacy rights and consumer protection not only for residents of California but for the whole nation as the law is frequently presented as a model for a federal data privacy law.
In that sense, the CCPA has the potential to become as consequential as the GDPR.
In July 2018, Bradford L. Smith, Microsoft’s president, compared the face recognition technology to products like medicines that are highly regulated, and he urged Congress to study it and oversee its use.“We live in a nation of laws, and the government needs to play an important role in regulating this technology,” Mr. Smith wrote.
More recently, in May 2019, U.S. Rep. Alexandria Ocasio-Cortez voiced her "absolute" concerns in a recent Committee Hearing on facial recognition Technology (Impact on our Civil Rights and Liberties).
Facial recognition bans (San Francisco and Somerville)
Privacy and civil rights concerns have escalated in the country as face recognition gains traction as a law enforcement tool and on 6 May 2019, San Francisco voted to ban facial recognition.
It is the first ban of its kind on the use of face recognition.
The anti-surveillance ordinance signed by San Francisco's Board of Supervisors would bar city agencies, including San Francisco PD, from using the technology as of June 2019. Yes, this includes law enforcement.
As reported by the Boston Globe of 27 June 2019, the Somerville City Council (Massachusetts) voted to ban the use of facial recognition making the city the second community to take such a decision.
Since the San Francisco and Sommerville rulings, the debate gets louder in many municipalities.
- Should other cities follow this example?
- Is the ban just a "pause button" to better assess risks?
- Is this a step backwards for public safety?
So, stay tuned for the outcome of all these discussions in the U.S.
More on biometric data protection laws in 2019 (EU, UK and US perspective) in our biometric data dossier.
India and its national biometric identification scheme, Aadhaar
In India, thanks to the Puttaswamy judgment delivered on 27 August 2017, the Supreme Court has enshrined the right to privacy in the country's constitution. This decision has rebalanced the relationship between citizen and state, and poses a new challenge to expansion of the Aadhaar project in 2018.
government however approved the use of the country's biometric EID program by private entities on 28 February 2019.
Rebound effect: the legal system and its professions get even stronger. As both ambassadors and guardians of data protection regulation, the post of data protection officer has become necessary for businesses and a much sought-after role.
#6 The rebels – facial recognition hackers
Despite this technical and legal arsenal designed to protect data, citizens and their
anonymity, critical voices have still been raised. Some parties are concerned and alarmed by these developments.
- In Russia, Grigory Bakunov has invented a solution to escape the eyes permanently watching our movements and
confuse face detection devices. He has developed an algorithm which creates special makeup to fool the software. However, he has chosen not to bring his product to market after realizing how easily it could be used by criminals.
- In Germany, Berlin artist Adam Harvey has come up with a similar device known as
CV Dazzle, and is now working on clothing featuring
patterns to prevent detection. The hyperface camouflage includes patterns in fabric such as eyes and mouthes in order to fool the face recognition system.
- In late 2017, a Vietnamese company successfully used a
mask to hack the Face ID face recognition function of Apple's iPhone X. However, the hack is too complicated to implement for large-scale exploitation.
- Around the same time, researchers from a German company revealed a hack that allowed them to bypass the facial authentication of Windows 10 Hello by printing a facial image in infrared.
- Forbes announced in an article from May 2018 that researchers from the University of Toronto have developped an
algorithm to disrupt facial recognition software (aka privacy filter).
In short, a user could apply a filter that modifies specific pixels in an image before putting it on the web. These changes are imperceptible to the human eye, but are very confusing for facial recognition algorithms.
The industry is working on
anti-spoofing mechanisms and two topics have been specifically identified by standardization groups :
- Make sure the captured image has been done from a person and not from a photograph (2D), a video screen (2D) or a mask (3D), (liveness check or
- Make sure that facial images (morphed portraits) of two or more individuals have not been joined into a reference document such as a passport.
#7 Further together – towards hybridized solutions
It's clear that the identification and authentication solutions of the future will borrow from all aspects of biometrics. This will lead to "biometrix" or a biometric mix capable of guaranteeing total security for all stakeholders in the ecosystem.
It's very much the spirit of Gemalto Assurance Hub, a risk assessment and fraud detection software for payments, where geolocation, ip-addresses (the device being used) and keying patterns can create a strong combination to securely authenticate users for on-line banking or egovernment services.
This seventh trend belongs to us.
It's our job to envisage it together, and make it happen through high-added-value biometric projects.
Face recognition and you
Now it's your turn.
The months to come hold many changes in store. Certainly we can't claim to predict all the important topics that will emerge in the short term future. Can you fill in some of the gaps?
If you've something to say on face recognition, tech or trends, a question to ask, or have simply found this article useful, please leave a comment in the box below. We'd also welcome any suggestions on how it could be improved, or proposals for future articles.
We look forward to hearing from you.