Reading time: 5 minutes
The European Union’s pioneering regulation, the Artificial Intelligence Regulation (“AI Act”), is not merely another administrative obligation for businesses, but the world’s first comprehensive legal framework that, in the context of the development and use of artificial intelligence, explicitly places the protection of fundamental rights and human dignity alongside – and, where appropriate, ahead of – technological progress. In a previous article in this series, we presented how, to this end, the AI Act applies the so-called risk-based approach; that is, rather than classifying the technology itself, it examines its specific use cases, categorizing AI systems into four different risk categories.
In the third part of our series, we use practical examples to illustrate the AI-based solutions that pose an unacceptable risk to fundamental rights and EU values and are therefore classified as prohibited AI systems under the AI Act.
General rules
First and foremost, it is important to note that the provisions regarding prohibited AI systems are among the rules that came into force on 2 February 2025. Accordingly, with a few exceptions, the placing on the market, putting into service, and use of prohibited AI systems are all prohibited within the EU.
Accordingly, the development and use of prohibited AI systems constitute a serious violation of the law, which may result in significant penalties. The fines imposed may amount to up to 7% of the company’s global annual turnover or 35 million euros.
List of prohibited AI-systems
In the following, we present the prohibited areas specified in the AI Act, illustrated with practical examples:
Subliminal, deliberately manipulative, or deceptive techniques
The prohibition extends to the use of so-called subliminal stimuli, i.e., those that operate below the threshold of consciousness or circumvent it. The essence of these mechanisms is that they operate covertly, thereby bypassing the conscious processing and rational defence mechanisms of the person concerned. An example of such a technique is when an AI system inserts visual or textual elements that appear for an extremely brief moment during a video playback; these elements are formally perceptible but are visible for such a short time that they cannot be consciously perceived, yet they are still capable of influencing the user’s attitudes or behaviour.
Targeted manipulative techniques that seek to distort decision-making through the use of sensory or psychological methods also fall within the prohibited category. This category also includes systems that deliberately use background sounds or visual elements to induce emotional or mood changes, thereby influencing users’ decisions.
Similarly, deceptive techniques aimed at restricting an individual’s autonomy, free will, or freedom of choice in such a way that the person is unaware of the influence being exerted or is unable to adequately control its effects are also prohibited. An example of such a practice would be an AI-powered chatbot that uses a synthetic voice to impersonate another person, such as a family member or friend, thereby deceiving the user and enabling the deployer to obtain an undue advantage or cause harm.
AI Systems exploiting vulnerabilities
AI systems that exploit the vulnerabilities of certain individuals or groups – such as those arising from age, disability, or social or economic situation – and are therefore particularly capable of enabling manipulative or exploitative practices are prohibited.
An example of such a system would be an AI system that targets elderly individuals with deceptive, personalized offers or fraudulent schemes, exploiting potential cognitive decline. The system aims to induce them to make decisions they would not otherwise make, and which are likely to result in significant financial harm.
Social scoring
The use of so-called social scoring systems is also prohibited, i.e. where an AI system evaluates the trustworthiness of individuals based on their social behavior or personal characteristics and assigns adverse consequences accordingly.
An example of this would be a private credit institution using an AI system to assess customers’ creditworthiness and determine eligibility for a mortgage loan based on personal characteristics that are not directly related to the purpose of assessing creditworthiness.
Individual risk assessment and prediction of criminal offenses
The AI Act prohibits AI systems that, based solely on profiling or the assessment of personality traits or individual characteristics, seek to assess or predict whether a natural person is likely to commit a criminal offence.
An example of this would be a public authority using a system that infers or predicts the risk of criminal behaviour – such as the commission of a terrorist act – based on characteristics such as age, nationality, address, type of vehicle, or marital status.
Establishment of facial recognition databases
The AI Act prohibits AI systems that create or expand facial recognition databases through the targeted but non-directed scraping of facial images from online sources or closed-circuit television (CCTV) footage.
Accordingly, it is prohibited, for example, for a company developing facial recognition software to use automated tools to collect facial images from social media platforms such as Facebook, YouTube, or other services. In such cases, the system searches for human faces among publicly available images on the internet and uses the collected material to build or expand a facial recognition database.
Emotion-recognition systems in the workplace and in schools
The AI Act prohibits AI systems from inferring the emotions of natural persons in workplace or educational contexts, except where such use is justified on medical or safety grounds.
For instance, it is not allowed for an employer to use technology that analyses facial expressions, body posture, or voice to infer employees’ emotional states, and to use such information for performance assessment or HR-related decision.
AI systems used for biometric categorisation of individuals
The AI Act prohibits biometric categorisation systems that individually categorise natural persons based on their biometric data in order to infer or deduce their race or ethnic origin, political opinions, trade union membership, religious or philosophical beliefs, sex life, or sexual orientation.
For example, it is prohibited to deploy an AI system that categorises users of a social media platform based on biometric analysis of photographs uploaded to the platform, in order to infer their presumed political orientation and deliver targeted political messaging.
AI systems for real-time remote biometric identification in publicly accessible spaces for law enforcement purposes
As a general rule, the AI Act prohibits the use of real-time remote biometric identification systems in publicly accessible spaces for law enforcement purposes. These systems identify individuals remotely without their active participation by comparing their biometric data against a database. Real-time operation means that the comparison and identification occur either simultaneously with the capture of biometric data or with only minimal delay.
An example of such a system would be surveillance cameras installed in metro stations for security purposes, which are connected to systems capable of identifying suspects in real time based on a reference database.
The rationale for this prohibition lies primarily in the risk of technical inaccuracies, which may lead to false or distorted results and potentially discriminatory outcomes. In addition, such systems may significantly affect the privacy of the general population and create a perception of continuous surveillance.
However, the AI Act also provides for specific exceptions, for example in cases such as the search for missing persons, the identification of victims of kidnapping or human trafficking, or situations involving sexual exploitation, where the use of real-time biometric identification may be permitted under strictly limited conditions.
Summary
The risk-based approach of the AI Act establishes a clear and coherent framework for the regulation of artificial intelligence in the European Union. Its core principle is that the legislator does not assess the technology itself, but rather its specific use cases and societal impacts. As a result, AI systems that pose an unacceptable risk to fundamental rights and EU values are subject to a comprehensive prohibition.
Photo source: pexels.com, panumas nikhomkhai