Who Does a Rejected Citizen Appeal To — The Empty Seat in Algorithmic Administration

A letter arrives. "You are a benefits fraudster. We are reclaiming the full amount you received." The person holding it has no idea why. They forged no document, hid no income. Yet nowhere on the letter is the name of an official who made the call. Because no person made it — an automated system assigned the score.

Over the past decade, about 26,000 parents in the Netherlands and roughly 470,000 debt notices in Australia were issued this way. Both countries handed the judgment at the core of their welfare and tax administration to automation, and both broke. But the way they broke was different. What was the same was the seat that broke. When a rejected citizen asked, "Why me, of all people?", there was no one left anywhere in the chain to take the question.

Two Countries That Broke at the Same Seat

The Dutch tax administration (Belastingdienst) ran a self-learning risk-classification model to screen for childcare-benefit fraud. The model scored each applicant, and the high scorers were pulled out for investigation. To be precise, the model automated the entrance to judgment — whom to suspect — and the final clawback decision came afterward, from a broken human process. When that entrance was wrong, about 26,000 parents were branded fraudsters and forced to repay their benefits in full. The number of households the government would later officially recognize as victims passed 33,000.

The fallout climbed to the top. In December 2020 a parliamentary inquiry committee branded the affair "unprecedented injustice" (Ongekend onrecht) and a violation of the fundamental principles of the rule of law, and the Rutte cabinet resigned in January the next year to take responsibility. A national government, in effect, fell because of scores an algorithm had assigned wrongly.

Australia's "Robodebt" worked differently from the start. It was neither a self-learning model nor a risk score. It was plain arithmetic that mechanically matched welfare records against annual income from the tax office and auto-calculated debts by "income averaging." Dividing an annual salary across 52 weeks to estimate weekly income, it counted someone who had worked only a few months of the year as if they had worked all of it, and stamped a debt. In 2019 the Federal Court ruled the method had "no legal basis" and was unlawful, and in 2023 a Royal Commission concluded in its final report that it was "a crude and cruel mechanism, neither fair nor legal … a costly failure of public administration, in both human and economic terms."

Two administrations that collapsed under automation

	Netherlands (toeslagenaffaire)	Australia (Robodebt)
What was delegated	Fraud risk score (self-learning model)	Auto-calculated welfare debt (income-averaging arithmetic)
Stage automated	Selecting whom to suspect (the entrance)	The debt decision (near-fully automated)
Period in use	2013–2018	2016–2020
Scale of harm	~26,000 parents misclassified, 33,000+ households officially recognized as victims	~470,000 debt notices
Legal verdict	DPA (AP): discriminatory and unlawful, €2.75M fine	Federal Court: unlawful (2019); Royal Commission: "crude and cruel"
The political bill	Cabinet resignation (Jan 2021)	Class-action settlement of A$1.872 billion

Sources: Dutch parliamentary inquiry · Data Protection Authority (AP) · Statistics Netherlands / Australian Federal Court · Robodebt Royal Commission (2019–2023)

Why did such different methods produce the same result? The two systems broke the same two things by different routes. Australia inverted the burden of proof; the Netherlands automated discrimination. And both emptied the seat where a rejected citizen could appeal.

Where the Burden of Proof Flipped — Australia

In ordinary administration, to brand a citizen a fraudster the state must prove the fraud. Australia, too, used to collect income data directly from employers to verify a debt. Robodebt reversed that order. Once the system issued a debt, the burden of proving it wrong shifted to the recipient. Unless you dug up pay slips and bank statements from years earlier and proved "I did not earn that much," the debt stood. The presumption of innocence had become a presumption of guilt.

This is not to say welfare fraud does not exist. Fraud detection is necessary in a tax-funded welfare system, and resources are finite. The problem is that, for the efficiency of detection, the burden of proof itself was dumped wholesale onto citizens — and onto the very people least able to produce documentation. Precarious workers and low-income recipients took this inverted burden first. The state keeps the efficiency; the weakest bear the cost of proof.

Where Discrimination Was Learned — the Netherlands

The Dutch model used (dual) nationality and foreign-sounding names as variables that raised the risk score. In a 2021 report, Amnesty International stated flatly that nationality was one of the risk factors in the assessment, and that this led to discrimination and racial profiling. The same structure surfaced in the city of Rotterdam's welfare-fraud prediction algorithm. When an investigative-journalism consortium opened the model up in 2023, what drove the score was ethnicity, gender, age, and Dutch-language ability — and young single parents with weak Dutch were the ones mostly summoned for investigation.

Here a common misreading needs correcting. This discrimination was not hand-designed by someone instructing the system to "suspect foreigners more." The Rotterdam model learned from the records of 12,707 past fraud investigations. If, in the past, human officials suspected foreigners and the poor more often, that bias sits in the data and is absorbed straight into the model. The algorithm did not invent discrimination — it learned the bias of past administration and set it like a specification. That is what makes it more dangerous. Human prejudice is erratic; a model's bias is fast, consistent, and aimed at the same people every time.

The belief that automation is more accurate than people wobbles too. Michigan's unemployment-fraud detection system (MiDAS) ran for two years with virtually no human oversight, and an audit found an error rate of about 93% among the determinations it reviewed. About 40,000 people were wrongly flagged as fraudsters. Yet rushing from there to "rip out all automation" is a mistake. Paradoxically, Rotterdam's discrimination came to light precisely because the model could be opened and audited. A human caseworker's private prejudice cannot be inspected; a model's weights can be verified. Opacity is often not a technical fate but an organization's choice to withhold. Auditability is clearly a strength of automation. It just does not stand in for a person who answers to the rejected individual. The Dutch court's 2020 order halting another fraud-detection system (SyRI) as a violation of the European Convention on Human Rights sits on the same line.

Where Did the Answerer Go?

Up to here this is a story of "the system was wrong." The real question is what comes next. Whom does a citizen meet when they try to appeal a wrong decision?

Where an outsourced model was used, as in the Netherlands and Rotterdam, responsibility slides. Go to the caseworker and you hear, "The system judged it that way." Go to the firm that built the system and you hear, "We just built it to the client's specification." The model itself only outputs a score and never says why. As responsibility runs downhill from official to vendor, from vendor to model, no one is left anywhere in the chain to say, "I judged you that way."

Australia took a different path. With no vendor and no model, there was no external object for responsibility to slide onto. Instead the answerer was hidden behind bureaucratic denial. There were senior officials and ministers who pressed ahead even knowing, from internal legal advice, that the scheme was likely unlawful. The difference shows at the end. After nearly a year of inquiry, the Royal Commission delivered 57 recommendations along with referrals of responsible individuals for criminal and civil action, placed in a separate sealed section. The hidden answerer was named again, by the force of an inquiry.

That contrast is the point. Delegation does not make the answerer evaporate on its own. The answerer is a seat that can be left empty. Scatter it across an outsourcing chain and it empties; send in a powerful inquiry and it fills again. The empty seat is not a fate of technology but a matter of design and will.

So what was the cabinet resignation? Political responsibility was indeed taken. But that is a collective, symbolic answer. A cabinet resigning is not an event that told one wrongly-clawed-back parent, "We got your case wrong." While political responsibility is processed on stage, the seat for answering each citizen's appeal one to one can still sit empty.

This structure is not unfamiliar. When the final judgment to kill is handed to a machine, a death occurs and the name of anyone to say "I did this" disappears — the same shape as the accountability gap in delegating lethal force. The difference is that in administration the empty seat arrives not on a battlefield but as a single letter, and can be filled again when an inquiry works.

The Bill Arrives Late, and Larger

What delegation emptied is not only the answerer. The cost did not vanish either — it only changed seats. At first it looked like efficiency. Automate what people used to screen one by one, and labor costs seemed to drop out. The bill for those savings came back, on a lag, in a larger sum.

Australia's government reached a settlement totaling A$1.872 billion in the class action. It refunded about A$721 million, paid A$112 million in compensation, and wiped the remaining debts in full. Per reporting, an appeal added further compensation, and as of 2026 the bill is still arriving. In the Netherlands, the data protection authority fined the tax administration €2.75 million for discriminatory data processing — and an incomparably larger cost was billed as the cabinet's resignation and as trust in administration itself. The cost pulled forward as efficiency came back as damages and legitimacy.

The heaviest bill is not counted in money. In Australia it was reported that some recipients of debt notices took their own lives, and the Royal Commission heard testimony of the severe distress, financial hardship, and deaths recipients suffered. The direct causal link between those deaths and the system, however, has not been officially established.

So: Korea, and You

If this still sounds like an accident in a far-off country, one thing has to be said. The same kind of automated judgment is already in our daily lives — in credit scoring, insurance underwriting, hiring screens, and the selection of audit targets. When your credit score drops a notch and a loan is refused, what can we ask of the model that set it?

Korea, too, has begun to answer this question by statute. Article 37-2 of the Personal Information Protection Act, in force since March 2024, gives the data subject the right to refuse a fully automated decision and to demand an explanation when it significantly affects their rights or duties. In the credit field, earlier still, a response right lets people demand an explanation of an automated-assessment result and ask for it to be recomputed. Europe goes one step further: it classed AI used for public-assistance eligibility and credit assessment as "high-risk," and those obligations apply from August 2026. What the law is belatedly trying to install is precisely that emptied seat — a person to answer the rejected citizen.

But stop here and you have seen only half. Article 22 of Europe's GDPR, the prototype of this refusal right, guaranteed the right to demand human intervention back in 2018. And yet Rotterdam's discriminatory model ran until 2023. Even when the law builds a seat for the answerer, if the person in it merely rubber-stamps the model's output, the seat stays empty. When "human intervention" is a rubber stamp, delegation erases the answerer again, one layer deeper.

So the question to ask when automating administration — and the screening in our own daily lives — is not "how efficient is it?" It is whether the seat where a rejected person can ask why is filled with a real person who answers. Leave that seat empty and take only the efficiency, and the cost does not disappear. It is billed to the weakest first, and to all of us later, in a larger sum.