EU AI Act
The EU AI Act is the first broad, binding, horizontal regulation of AI systems. Unlike the NIST AI RMF, which you may adopt or ignore, this one carries legal obligations and enforcement. For an engineer the important insight is that it is risk-based, not technology-based: nothing in it says “large language model must do X.” It classifies systems by what they are used for and attaches obligations to that classification. The same model can be unregulated in one product and high-risk in another.
This page is a map of the structure and a translation of the obligations into things you actually build. It is deliberately vague about article numbers and dates, because the parts of this regulation that engineers get wrong are structural, not numerical.
The risk-based structure
| Tier | Rough meaning | What it means for you |
|---|---|---|
| Prohibited practices | A short list of uses considered unacceptable | Do not build it. No amount of controls makes it compliant. |
| High-risk systems | Uses with significant potential impact on safety or fundamental rights | The heavy obligations regime; most engineering effort lands here |
| Transparency-obligation systems | Systems that interact with people or generate/manipulate content | Disclosure and marking duties, comparatively light but mandatory |
| Minimal risk | Everything else | No specific obligations; voluntary codes encouraged |
Prohibited practices cover things like certain manipulative techniques that exploit vulnerabilities, certain social-scoring uses, and certain biometric applications. The exact list and its carve-outs are narrow and heavily qualified — read the text, do not work from a blog summary.
High-risk is reached by two broad routes: the AI system is a safety component of a product already covered by EU product-safety legislation, or the system falls into an enumerated set of sensitive use areas (employment and worker management, education access, essential services and creditworthiness, law enforcement, migration and border control, administration of justice, and critical infrastructure among them). There are conditions under which a system in an enumerated area is not treated as high-risk because it performs only a narrow preparatory or procedural task — a genuinely important escape hatch that requires documented justification rather than an assumption.
Transparency obligations attach independently of tier: telling people they are interacting with an AI system, disclosing emotion-recognition or biometric-categorisation use, and marking synthetic or manipulated content in a machine-readable way. For agentic products this is the tier most teams actually touch first — a customer-facing agent needs to say it is an agent.
General-purpose AI models: a separate regime
Alongside the system-level risk tiers, the Act imposes a distinct set of obligations on providers of general-purpose AI (GPAI) models — foundation models that can serve many downstream purposes. These obligations are about the model itself rather than a use case, and center on technical documentation for the model, information provided to downstream providers who integrate it, a policy respecting EU copyright law, and a sufficiently detailed summary of training content. Models judged to carry systemic risk carry additional duties around evaluation, adversarial testing, incident reporting, and cybersecurity protection of the model.
Why this matters even if you will never train a foundation model: when you integrate a GPAI model into your product, you are a downstream provider, and the documentation the model provider owes you is exactly the input you need for your own technical documentation. If a vendor cannot give you model documentation adequate for your obligations, that is a procurement red flag, not a paperwork inconvenience.
Provider vs deployer — the distinction engineers get wrong
Most engineers assume “we did not train the model, so the obligations are the model vendor’s problem.” That is usually wrong.
| Role | Roughly who | Obligation weight |
|---|---|---|
| Provider | Develops an AI system, or has one developed, and places it on the market or puts it into service under its own name or trademark | Heavy: the full high-risk regime attaches here |
| Deployer | Uses an AI system under its own authority, in a professional context | Lighter but real: use per instructions, human oversight, input-data relevance, log retention, informing affected people |
| Importer / distributor | Brings a third-party system into or through the EU market | Verification and due-diligence duties |
Two traps:
- Building on someone else’s model usually makes you the provider of your system. If you ship an agent product under your brand, you are the provider of that system even though the weights are someone else’s. The model vendor’s GPAI obligations do not discharge yours.
- A deployer can become a provider. Substantially modifying a high-risk system, putting your own name on it, or changing its intended purpose in a way that makes it high-risk can shift the provider obligations onto you. Fine-tuning, re-prompting into a new domain, or wiring a general assistant into a hiring workflow are the kinds of changes that trigger this discussion.
Decide your role explicitly, write it down with the reasoning, and revisit it whenever the intended purpose changes. Getting this wrong is the difference between a two-page deployer checklist and a full conformity assessment.
Extraterritorial reach
The Act’s scope is not limited to companies established in the EU. It reaches providers placing systems on the EU market regardless of where they are established, and — critically — it can reach providers and deployers located outside the EU when the output produced by the system is used in the EU. A US company running an agent that produces decisions or content consumed by users in the EU can be in scope even with no EU entity.
The practical consequence: “we are not a European company” is not a classification answer. The questions are where your users are, where the output lands, and whether you place the system on the EU market. Answer those before you conclude you are out of scope.
High-risk obligations translated into engineering
This is the core translation table. The left column is the regulatory concept; the middle is what you build; the right is what you show an assessor.
| Obligation area | What you actually build | Artifact that proves it |
|---|---|---|
| Risk management system | A continuous, documented risk process across the lifecycle — not a one-time review | Risk register with versions, owners, residual-risk decisions, review dates |
| Data and data governance | Documented provenance, quality checks, and bias examination for training/validation/test and for retrieval corpora | Dataset datasheets, ingestion pipeline configs, data-quality test results |
| Technical documentation | A maintained description of design, architecture, capabilities, limitations, and controls | Versioned tech-doc pack generated from repo sources, not hand-written yearly |
| Record-keeping / logging | Automatic event logging over the system’s lifetime with defined retention | Trace store with retention policy and integrity guarantees |
| Transparency to deployers | Instructions for use: intended purpose, limits, oversight measures, expected accuracy | Published usage documentation, versioned with the release |
| Human oversight | Design that lets a competent human understand, intervene, override, and stop | Approval gates, override UI, documented oversight procedure, training records |
| Accuracy, robustness, cybersecurity | Declared performance metrics, resilience to error and adversarial manipulation | Eval reports, adversarial suite results, pen-test and threat-model records |
| Conformity assessment | Demonstrating requirements are met before market placement, plus declarations and marking | Assessment file, declaration of conformity, registration records |
| Post-market monitoring | Ongoing collection and analysis of real-world performance, plus serious-incident reporting | Monitoring plan, production metrics dashboards, incident log with timelines |
| Quality management system | Organizational processes ensuring the above happen repeatably | Documented procedures, roles, and internal audit records (see ISO/IEC 42001) |
Three of these map almost one-to-one onto agent engineering work you should be doing anyway.
Logging and traceability
For an agent, “automatic logging” cannot mean application logs. It means a per-request trace that records the decision chain: which model version and prompt template ran, which tools were called with which arguments, what each tool returned, which authorization decisions were made, and where a human intervened. Two properties make it evidence rather than debugging output: completeness (you can reconstruct any past decision) and integrity (the record cannot be quietly edited). Append-only storage, a defined retention period tied to your obligations, and an argument digest instead of raw arguments for sensitive fields are the usual design. The implementation is in Evidence Automation and Lab 4.
Human oversight
Oversight is a design requirement, not a policy sentence. To be meaningful, the human in the loop needs: enough context to judge the action (not just “approve?”), the authority and the mechanism to override or stop, awareness of automation bias, and a record that they actually decided. The design questions worth arguing about are which actions require approval (irreversible and high-value ones), whether approval is blocking or post-hoc, what happens on timeout — and defaulting to deny on timeout is the only defensible answer for irreversible actions.
A useful framing for the oversight design conversation:
| Action class | Example for a support agent | Oversight design |
|---|---|---|
| Reversible, low value | Draft a reply, look up an order | Fully autonomous; logged only |
| Reversible, high volume | Tag and route a ticket | Autonomous; sampled review with a quality metric |
| Irreversible, bounded | Refund under a cap to the matching customer | Post-hoc review plus a hard policy cap |
| Irreversible, unbounded | Refund over cap, account closure, external email | Blocking human approval, deny on timeout |
Cybersecurity
The Act treats cybersecurity as a requirement for high-risk systems, and for agents the AI-specific attack surface is the interesting part: prompt injection through retrieved content and tool outputs, tool misuse, data exfiltration through generated output, and model or supply-chain tampering. Your evidence here is a threat model referencing a recognized taxonomy plus test results against it — see OWASP LLM Top 10 and MITRE ATLAS. A generic infrastructure pen-test does not cover this and should not be presented as if it does.
Where to start on a real product
A sequence that produces useful output in a week rather than a quarter:
- Write the use-case paragraph. Deployment context, decision influenced, people affected, geography of the output.
- Determine your role. Provider, deployer, or both, with the reasoning written down and a trigger list for re-evaluation.
- Classify the tier. Prohibited, high-risk, transparency-obligation, or minimal — and if the answer is “probably not high-risk,” record why, because that reasoning is the artifact.
- Gap-assess against the translation table above. For each row, mark: exists and evidenced, exists but not evidenced, or missing. The middle category is usually the biggest and the cheapest to fix.
- Fix the evidence gaps first. Teams routinely already do risk management, evals, and oversight, but produce nothing durable. Wiring traces and approval records is engineering work you control, unlike legal interpretation.
- Escalate the genuine ambiguities to counsel with a specific question, not a general one.
How this fits the other frameworks
The Act tells you what outcome is required. It does not tell you how to organize the work. NIST AI RMF gives you a way to structure the risk process, and ISO/IEC 42001 gives you a management system that produces the procedural evidence an assessment will want. Harmonised standards developed for the Act are intended to give a presumption of conformity when followed, so tracking which standards are being developed for which requirement is worth doing — again, from the primary source.
A reasonable stack for an agent product: classify and scope with the AI Act, run the risk process with the AI RMF functions, operate it as a management system aligned to ISO/IEC 42001, and generate the evidence continuously as described in Evidence Automation.
Misconceptions worth correcting
These come up in almost every first conversation with an engineering team.
| Misconception | Reality |
|---|---|
| “It only regulates large language models.” | It regulates AI systems by use case, and includes a separate regime for general-purpose models. Simple statistical systems in a sensitive use area can be in scope. |
| “We use a vendor model, so it is their compliance problem.” | Their GPAI obligations are about the model. If you ship a system under your name, the system-level obligations are yours. |
| “We are not in Europe.” | Scope can follow where the output is used, not only where the provider is established. |
| “High-risk means we cannot ship.” | High-risk means a defined set of obligations must be met before placing on the market. It is a compliance path, not a ban — prohibited practices are the ban. |
| “One classification, done forever.” | Changing intended purpose, adding tools that change what decisions the system influences, or substantially modifying it can change both tier and role. |
| “Logging means keeping application logs.” | It means traceable records adequate to reconstruct the system’s operation, retained for a defined period, with integrity. |
What to practice
The most transferable skill from this page is not reciting tiers. It is being able to take one real product and produce, in a single page:
- The use-case paragraph — deployment context, the decision influenced, who is affected, where the output lands.
- The role determination — provider, deployer, or both, with the reasoning and the events that would change it.
- The tier — with the reasoning written out, including why it is not a higher tier if you concluded that.
- A gap table over the obligation rows above, each marked evidenced, unevidenced, or missing.
That page is a deliverable a legal team can react to and an engineering team can plan from, which is exactly the translation work this section exists to teach. Practise it on a product you already know well, then on one you do not.
For the threat side of the cybersecurity requirement, continue to OWASP Agentic Threats; for the evidence side of logging and oversight, continue to Evidence Automation.