The Department for Work and Pensions (DWP) has revealed how it is using AI to identify potential overpayments and fraudulent claims across various benefits, including Universal Credit and Personal Independence Payment (PIP).
Sharing new details in the official government algorithm database, the department says it is using an AI tool called ‘intelligent claims analysis’ to flag cases that require further review.
Claimed by millions, Universal Credit and PIP are among the key benefits designed to support those in financial hardship or with long-term disabilities.
DWP states that the AI tool helps identify inconsistencies in claims by cross-referencing application data with past submissions and external databases, such as tax and employment records. A human agent then reviews flagged cases before any action is taken.
The system works by assessing key indicators within a claim, comparing them against historical data patterns, and assigning a “risk score”.
The DWP reports an 89% accuracy rate in predicting claims that require further scrutiny.
Once flagged, the case is registered in the department’s fraud and compliance system, prompting an additional manual review by a specialist. This ensures that decisions to suspend or deny payments are not made solely by AI but involve human oversight.
Despite being in use since mid-2021, the DWP publicly shared details of the tool for the first time this week.
According to the department, the system has processed over 1.2 million cases and saved approximately 60,000 operational hours by reducing unnecessary manual reviews.
However, the system’s early iterations were not without flaws. Initial versions from 2021 to 2023 reported a higher-than-expected rate of false positives, meaning legitimate claims were flagged incorrectly. Adjustments to the algorithm, incorporating a broader range of contextual factors, improved its predictive performance.
Concerns have been raised about the fairness and transparency of automation in benefit decision-making. Advocacy groups warn that AI-driven fraud detection systems could disproportionately impact vulnerable individuals.
James Walker, director at Welfare Rights UK, said: “While ensuring taxpayers’ money is used appropriately is important, we must also ensure that AI-driven fraud detection does not unfairly target those who genuinely need support. The DWP must provide clear explanations and appeal routes for those affected by AI-flagged claims.”
In December, a separate AI tool used by the DWP to detect welfare fraud was found to exhibit bias related to age, disability, and nationality. An internal review acknowledged that “statistically significant outcome disparities” were present in its findings, according to The Guardian.
However, the DWP states that the new claims analysis tool carries a low risk of bias as it does not use personal details such as name or ethnicity but focuses solely on claim patterns and financial data.
The department’s risk assessment of the tool also addresses concerns that automation could diminish human oversight, stating: “To mitigate this, staff have received training to critically assess flagged cases and ensure AI recommendations are not accepted without scrutiny.”
Details about the AI tool come as Keir Starmer continues Labour’s push to “turbocharge” AI adoption in public services. The Prime Minister has claimed that AI will help improve efficiency in welfare, healthcare, and education.
Despite this, the fraud detection tool is one of the few AI systems disclosed by the DWP on the algorithm transparency register, a requirement for all government departments for over a year. The department has confirmed that an internal inventory of AI tools exists but has resisted calls to publish the full list.
When asked in September to disclose the full range of AI systems under the Freedom of Information Act, the DWP responded: “Public authorities like the DWP must retain control over how and when they release information. The ability to manage disclosures is essential to the effective operation of government functions.”
The measures will likely be deployed with a focus on Universal Credit, as this has a large number of claimants, with 6.4 million claimants as of January 2024, reports the Express.
Officials could also target people on Jobseeker’s Allowance, Employment and Support Allowance, and Housing Benefit, given these benefits have historically had higher levels of irregularities.
