An organisation that uses an automated system is liable for the discriminatory outcomes it produces, whether or not it built the model and whether or not anyone intended the result. In the Netherlands that liability rests on tort law (Article 6:162 of the Dutch Civil Code, BW), equal treatment legislation and the GDPR; the high-risk rules of the EU AI Act for recruitment, credit and insurance systems apply only from 2 December 2027.
In Dutch discrimination law the effect of a decision counts, not the motive. A recruitment filter, credit model or fraud score that disadvantages a protected group can therefore lead to a civil claim, a complaint to the Netherlands Institute for Human Rights and an investigation by the Dutch Data Protection Authority, all on the same facts.
What is algorithmic bias and where does it come from?
Algorithmic bias arises when a system treats comparable people differently on a protected ground, or on a criterion that acts as a substitute for one. Because a model applies the same pattern to every file, a biased model works like a policy rather than an incident.
- Training data: a model trained on past decisions reproduces the patterns in them.
- Variables: a neutral-looking feature such as postcode can stand in for ethnicity and cause indirect discrimination.
- Use of the output: a score that is acceptable as advice can be unlawful as the sole basis for rejecting an application.
- Lack of monitoring: a model that was balanced at launch drifts as the data change.
Direct discrimination uses a protected ground openly. Indirect discrimination, where a neutral criterion disadvantages a protected group, can be justified only if the aim is legitimate and the means are appropriate and necessary, and the organisation must be able to prove that.
When is an organisation liable under Dutch law?
Two bodies of Dutch law do most of the work. Tort law asks whether the organisation acted carelessly and caused loss; equal treatment law asks whether the outcome was a prohibited distinction.
Tort: negligent deployment
Under Article 6:162 BW, a party is liable for an unlawful act (onrechtmatige daad) attributable to it that causes damage. Using a system that was not tested, validated or monitored can itself be careless conduct. The claimant must prove the unlawful act, attribution, damage and the causal link.
Buying the system from a supplier does not relieve the deploying organisation of this duty. It may have a separate claim against the supplier for non-performance (Article 6:74 BW), but that does not affect the claim of the rejected applicant.
Equal treatment law and the burden of proof
The General Equal Treatment Act (Algemene wet gelijke behandeling, AWGB), the age discrimination act (WGBL) and the act on disability and chronic illness (WGBH/CZ) prohibit direct and indirect distinction on protected grounds; in employment, Article 7:646 BW adds the rules on gender. Once the claimant puts forward facts that suggest discrimination, the organisation must prove that it did not discriminate.
An organisation that cannot show what its model weighed, on which data and with which testing, cannot meet that burden. A complaint can also go to the Netherlands Institute for Human Rights (College voor de Rechten van de Mens), whose opinions are not binding but are often used as evidence in later court cases.
What does the GDPR require of automated decisions?
Article 22 GDPR gives a person the right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects. Recruitment, credit and benefit decisions usually qualify.
Human involvement takes a decision outside Article 22 only if it is meaningful: a person with the authority and information to decide differently, not someone who confirms what the screen shows. The fairness principle in Article 5(1)(a) applies in any case, and Article 35 requires a data protection impact assessment before high-risk processing such as systematic profiling. Under Article 83(5) GDPR, fines can reach €20 million or 4% of worldwide annual turnover, whichever is higher. See our overview of il-privatezza tad-dejta, il-GDPR, l-IA u l-big data.
What does the AI Act require now, and what starts later?
The AI Act, Regulation (EU) 2024/1689, entered into force on 1 August 2024 and applies in stages. The prohibited practices in Article 5 apply since 2 February 2025; the high-risk obligations for recruitment, credit and insurance systems apply from 2 December 2027.
The obligations for general-purpose AI models and the penalty provisions apply since 2 August 2025. The Digital Omnibus on AI, Regulation (EU) 2026/1744 in force since 27 July 2026, postponed the high-risk regime: systems listed in Annex III, such as those used for recruitment, promotion and termination, creditworthiness and insurance risk assessment, follow on 2 December 2027, and high-risk systems in regulated products (Annex I) on 2 August 2028. The Omnibus also softened the AI literacy duty in Article 4 into a duty to support AI literacy among staff.
Once the high-risk regime applies, it requires representative and checked training data, technical documentation, logging, transparency, human oversight and a conformity assessment. Deployers have duties of their own, such as keeping logs and assigning oversight to competent staff. Fines reach €35 million or 7% of worldwide turnover for prohibited practices and €15 million or 3% for breaches of the high-risk obligations. The proposed AI Liability Directive has been withdrawn, so a damages claim in the Netherlands remains a claim under Dutch law. More on this in our guide to l-aspett legali tal-intelliġenza artifiċjali u l-Att tal-UE dwar l-IA.
What have Dutch courts and regulators decided?
On 5 February 2020 the District Court of The Hague ruled that the legislation behind SyRI, the government’s fraud risk system, breached Article 8 of the European Convention on Human Rights because it was insufficiently transparent and verifiable (ECLI:NL:RBDHA:2020:865). Neither the risk model nor the indicators were disclosed, so no one could check for discriminatory effects.
In the childcare benefits scandal (toeslagenaffaire), the tax administration used risk selection in which dual nationality counted as a fraud indicator. The Dutch Data Protection Authority found this processing unlawful and discriminatory and fined the tax administration. The lesson for private organisations: a system nobody can explain cannot be defended, and choosing to run it is the act that is judged.
Who is liable: the supplier or the user?
The person harmed claims against the organisation that took the decision, which is almost always the deployer. Under the AI Act, the provider and the deployer each have their own duties.
A contract with the supplier can allocate the financial consequences and secure the information you need to defend a claim, but it cannot remove a regulatory fine or a statutory duty. Our article on min hu responsabbli għall-iżbalji li jsiru mill-intelliġenza artifiċjali discusses that allocation.
What should you check as an organisation using AI?
- Have you tested outcome rates across protected groups before deployment and checked variables for proxies?
- Do you keep testing after go-live, and has a named person the authority to suspend the system?
- Is human review meaningful where Article 22 GDPR applies, and was the impact assessment done before go-live?
- Does the works council have a right of consent under Article 27 of the Works Councils Act (WOR), for example for systems that monitor staff?
- Does the supplier contract give you access to documentation, test results and logs, and an audit right?
- Can you explain each decision to the person affected in plain language?
What should you check as an applicant or employee affected by a decision?
- Was the decision taken without meaningful human involvement? You can then invoke Article 22 GDPR.
- Have you asked for access to your data and an explanation of the logic involved (Article 15 GDPR)?
- Are there facts suggesting a protected ground or proxy played a role? The burden of proof then shifts.
- Is an opinion (oordeel) from the Netherlands Institute for Human Rights enough, or do you need a damages claim in court?
What can we do for you with algorithmic bias liability?
tagħna Avukati tal-IT advise companies on the legal side of automated decision-making.
- We assess your exposure under Dutch tort law, equal treatment law and the GDPR for a specific system.
- We review your impact assessment and human-oversight process for Article 22 GDPR.
- We prepare your AI Act compliance for high-risk recruitment, credit and insurance systems.
- We negotiate supplier contracts on documentation, audit rights and liability.
- We handle complaints, Institute procedures, claims and regulatory investigations.
sommarju
- The deploying organisation is liable for discriminatory outcomes, even with a bought-in system.
- Once facts suggest discrimination, the organisation must prove it did not discriminate.
- Article 22 GDPR restricts solely automated decisions; human review must be real.
- AI Act high-risk duties for recruitment, credit and insurance apply from 2 December 2027.
- Documentation of testing and monitoring is the core of any defence.
Mistoqsijiet frekwenti
If our third-party AI is biased, who is liable: the vendor or us?
Towards the person affected, usually you as the deployer. You may have a contractual claim against the vendor (Article 6:74 BW), but that does not protect you against a civil claim or a regulatory fine.
How do we prove our algorithm is not discriminatory?
With records made before and during use: data sources, validation tests across groups, regular audits, corrective steps and an explanation of the decision logic. For high-risk systems under the AI Act, such documentation becomes mandatory from 2 December 2027.
Does explainable AI remove our liability?
No. Explainability helps meet GDPR transparency duties, but if the explanation shows that a protected ground or a proxy decided the outcome, the decision is still discriminatory.
Do these rules apply to SMEs?
Yes. Tort law, equal treatment law and the GDPR apply regardless of size. The AI Act eases some burdens for SMEs, but high-risk duties for recruitment or credit systems still apply.
