The report from the European Commission’s High-Level Expert Group on AI listed a number of areas of AI development that pose risks, including the identifying and tracking of individuals; covert AI systems; AI-enabled ‘citizen scoring’; and lethal autonomous weapon systems (LAWS).

Use of AI to track people should “uphold the autonomy of European citizens”, the report argues.

It states that it must be clearly defined when and how AI can be used to automatically identify people and differentiate between mere identification and tracing and tracking as well as between targeted surveillance and mass surveillance.

Legally, a means of consent must also exist for individuals to submit to this kind of scrutiny.

The report also warned that humans should always know if they are directly interacting with another human being or a machine and that AI should not be covertly used.

“It should be borne in mind that the confusion between humans and machines could have multiple consequences such as attachment, influence, or reduction of the value of being human.”

Scoring of citizens “in all aspects and on a large scale by public authorities” endangers the freedom and autonomy of citizens, the report also states.

A Chinese app announced in January this year allows people to know when they are in the vicinity of debtors. Its name when translated into English is a “map of deadbeat debtors”.

On the danger of autonomous weapons, the report states their development could lead to an “uncontrollable arms race on a historically unprecedented level, and create military contexts in which human control is almost entirely relinquished and the risks of malfunction are not addressed.”

“The European Parliament has called for the urgent development of a common, legally binding position addressing ethical and legal questions of human control, oversight, accountability and implementation of international human rights law, international humanitarian law and military strategies.”