FCA warns AI governance gaps risk speedier consumer harm, urges expanded incident reporting

The UK’s financial regulator is urging firms to report significant failures and near-misses involving artificial intelligence systems. This directive extends existing incident reporting rules to cover autonomous agents, addressing the specific risks these tools pose to financial stability and consumer protection.

Jessica Rusu, the Financial Conduct Authority’s chief data, information and intelligence officer, outlined this position at Bloomberg’s UK Policy Series on Monday. She argued that as financial services adopt increasingly autonomous AI agents, clear accountability becomes essential. The regulator aims for Britain to lead in safe AI adoption within the sector.

What we know

Rusu warned that poorly governed AI can create consumer harm, cyber vulnerabilities, and operational failures. She noted these issues could occur at much greater speed and scale than traditional software errors. Consequently, she stated that current rules requiring major incident reports must extend to significant AI failures and near-misses. This expansion is necessary to maintain oversight of autonomous systems.

The UK artificial intelligence minister, Kanishka Narayan, supported this view last week. He told MPs that the government plans to work with Parliament and technical experts. Their goal is to assess how to strengthen the regime against AI-driven cyber threats and loss-of-control risks. This includes identifying and reporting serious incidents across essential services.

Steven Adler, co-founder of Guidelight AI Standards, also spoke to MPs last week. He suggested that AI oversight could learn from the aviation industry. In most jurisdictions, aviation requires mandatory reporting of near-misses. Adler implied that a similar approach could help regulators manage the risks associated with autonomous financial agents.

Defender's checklist

• Review current incident reporting procedures to ensure they capture AI-specific failures.
• Identify all autonomous or semi-autonomous AI agents currently in use within operations.
• Establish clear accountability lines for decisions made by AI systems.
• Monitor for operational anomalies that may indicate AI system drift or failure.

The background in plain language

A near-miss is an event that could have caused harm but did not, often due to intervention or luck. Reporting these incidents helps organisations identify weaknesses before a real failure occurs. Autonomous AI agents make decisions without constant human supervision. Because they act independently, errors can spread quickly. Tracking near-misses allows teams to spot patterns, adjust controls, and prevent minor glitches from becoming major systemic disruptions in critical infrastructure.

Background: AI governance

Implement basic AI governance by mapping your data flows, selecting models with clear usage rights, and integrating checks into your workflow. You do not need large budgets; you need clear rules and consistent application of those rules across your team.

Read the full guide: AI Governance for Small Teams: Practical Controls and Cost

What to do and how to stay safe: FCA

• Audit your organisation’s AI usage to map where autonomous decisions are made.
• Update incident response plans to include criteria for reporting AI-related anomalies.
• Train staff to recognise and document unexpected behaviour from AI systems.
• Engage with industry groups to share best practices for AI risk management.

Step-by-step guide: Deepfake Fraud Response: Stop the Bleeding and Secure Recovery

General security guidance from the TechWire Beat newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Does the FCA currently mandate AI failure reporting?

The FCA states that existing rules for major incidents must extend to cover significant AI failures and near-misses.

Who announced this requirement?

Jessica Rusu, the FCA’s chief data, information and intelligence officer, announced it at Bloomberg’s UK Policy Series.

Why is this considered a security priority?

The AI minister stated that near-miss reporting for agentic AI is becoming a national security priority.