Growing backlash in China against A.I. and facial recognition

China’s seemingly unfettered push into facial recognition is getting some high-level pushback.

Face-swapping app Zao went viral last weekend, but it subsequently triggered a backlash from media — both state-run and private — over the apparent lack of data privacy protections.

The app also stirred controversy as it allows users to upload photos of themselves and superimpose their faces on clips of celebrities or anyone else. Its capability raised concerns that people could manipulate videos to spread disinformation.

“The future has come, artificial intelligence is not only a test for technological development, but a test for governance,” city newspaper The Beijing News, wrote Sunday in Chinese, according to a CNBC translation.

“Right now it’s very difficult to determine whether the software operator’s collection of human facial data and authorization are malicious, but netizens’ concerns are understandable,” it wrote.

That’s a marked shift from silence, or even straight-out commentary that Chinese citizens don’t care much about giving up data privacy for convenience.

The Zao app was able to quickly demonstrate the power of “deepfake” and influence on social media. Deepfake is the ability to manipulate videos or digital representations with the help of computers and machine-learning software, and make them appear to be real, when they are not. Many are increasingly worried about how the manipulated content could influence elections with false information.

In the case of Zao, the rapid proliferation of user-generated deepfakes reportedly prompted ubiquitous messaging app WeChat to ban users from sharing content created by the app. The incident is an example of a growing discussion in China about the safety of data collection and facial recognition-based payments.

Zao revised its privacy policy within a few days of its release following backlash from users.

Chinese authorities may already be signaling there’s a need to take a tougher stance on facial recognition technologies.

Citing the case of Zao, the Ministry of Industry and Information Technology said it’s necessary to “strengthen the security assessment of new technologies and businesses” and guard against “network fraud and other hidden dangers,” according to CNBC’s translation of its statement on social media platform, WeChat.

From a professional standpoint, it’s impossible for face-swapping technology to pose a security threat to facial recognition payment, Zao said in an online statement Tuesday.

If Zao sparks greater scrutiny on the security of facial recognition, Chinese authorities may tighten further in this area.

China’s delayed implementation of regulations — intentional or not — has helped some major technology companies flourish. Authorities have tended to be reactive rather than proactive in its regulation of new technologies.

For example, Chinese tech companies often warn in their prospectuses for going public in New York about their lack of licensing or the potential for regulation to change. In the recent case of peer-to-peer lending platforms in China, loose regulation allowed a few thousand start-ups to take customers’ money before a significant portion was lost. After an acceleration in the fallout last summer, Beijing banned new online lending platforms.

Just this summer, a senior director at China’s central bank warned consumers of the new risks posed by artificial intelligence and facial recognition technology.

“A bank card may still be in a pocket, but a face is usually exposed, and recognition is very easy. Current technology can recognize your face from three kilometers away,” said Li Wei, director of the science and technology department of the People’s Bank of China, said in July at a fintech summit in Beijing co-organized by the think tank China Finance 40 Forum.

Li also warned companies not to exploit their technological capacity, according to a CNBC translation of a Chinese-language transcript posted on the forum’s official WeChat account.

The People’s Bank of China did not immediately respond to a CNBC request for comment.

But the abuse of facial recognition data has far more serious consequences than a fintech lending platform stealing money — especially since the information is increasingly becoming an irrevocable way to verify identity or make a payment.

“If your password is compromised … you can just change the password. If that situation occurs with your face or fingerprint, the impact can be much longer lasting,” said Martin Chorzempa, research fellow at international economic policy firm Peterson Institute for International Economics.

“You can’t change your fingerprint or face like you can change a password,” he explained, therefore using biometric authentication can be more dangerous for users than using a traditional password.

The growing fear for many about facial recognition technology is how easy it is to collect and harvest mass biometric data.

In fact, one of China’s most popular online payment systems addressed the public’s growing concerns immediately.

Alipay, the payment system run by e-commerce giant Alibaba‘s affiliate Ant Financial, said Aug. 31 that facial recognition-based payment requires a three-dimensional face. The software and hardware will detect whether the face is an image, video or a software simulation, it said in a statement.

The tech firm said additional security measures, such as verification by mobile phone, are sometimes needed before paying, and assured customers it would bear the costs in cases of facial recognition payment fraud.

At the end of the day, Chorzempa said, “security for online transactions has always been an arms race.”

Zao’s quick response to the public’s outcry over its use of personal information is rare, said Ziyang Fan, head of digital trade at the World Economic Forum, in an email Tuesday.

“This backlash shows that the concept of data privacy is gaining ground in China, and increasingly users are less willing to trade privacy for convenience or entertainment,” he said.

“We may expect to see more heightened awareness from both users and companies in China on data privacy in the future,” Fan added.

In contrast to many other major countries, the Chinese government has often chosen to let companies build out new products before setting regulations.

In comparison, some major American cities have already banned the use of facial recognition. In March, Missouri introduced the Commercial Facial Recognition Privacy Act of 2019. The move was considered by some as an indication the federal government was willing to consider a nationwide ban on facial recognition technology.