Summary
What this article covers in 30 seconds
- Existing regulatory requirements continue to apply when firms use AI.
- Firms need visibility of AI use across customer interactions and operational processes.
- Human oversight must be informed, effective and supported by reliable evidence.
- Monitoring should identify poor outcomes, bias, inaccuracies and emerging risks.
- Third-party technology remains subject to the firm’s oversight and accountability.
Why does AI make effective oversight more important?
Firms need to understand the problem an AI system is intended to solve, the customers or processes it affects and the potential harms it could create. That requires more than an initial approval. Firms need ongoing visibility of how systems operate in practice and whether their outputs remain accurate, appropriate and consistent with the intended use.
The Consumer Duty is particularly relevant. AI-supported processes ought to be monitored across different customer characteristics and scenarios, including the experiences of customers in vulnerable circumstances. Firms should identify patterns that could indicate bias, inaccuracy or inconsistent treatment, rather than relying solely on small-scale manual checks or waiting for a complaint to reveal a problem. This is where effective monitoring and evidence become important. Firms need sufficient visibility of the interactions, decisions and outcomes affected by AI to assess whether controls are working in practice and identify where intervention may be needed.
What should meaningful human oversight look like?
Human oversight should not simply involve placing a reviewer at the end of an automated process. Reviewers need enough information, expertise and authority to challenge output and intervene when necessary. The level of oversight should reflect the risk of the use case. An AI tool used to summarise internal information is unlikely to require the same control environment as one influencing suitability assessments, customer communications or decisions with a direct financial impact.
Technology can support this oversight by giving reviewers access to the original interaction, the AI-generated output and the evidence behind the eventual decision. It can also help firms direct human attention towards higher-risk cases, exceptions and emerging patterns instead of relying entirely on broad sample-based monitoring.
How can firms evidence accountability?
Using AI does not remove individual or organisational accountability. Material use cases should have defined owners, appropriate controls and defined routes for escalation. An up-to-date AI inventory is a useful starting point. This might record each tool’s purpose, provider, owner, risk classification and connection to customer or regulatory outcomes. Material applications can then be mapped to accountable senior managers.
However, an inventory only shows what is intended to happen. Firms also need evidence of what happens in practice. That could include records of testing and approvals, changes to models or prompts, exceptions identified through monitoring, interventions by human reviewers and the action taken when an issue is found. Consistent evidence helps compliance teams assess the effectiveness of controls and gives senior management clearer information on the risks and outcomes associated with AI use.
Does using third-party AI change the firm’s responsibilities?
Using a third-party tool, model or platform does not transfer responsibility for customer outcomes. Firms should understand how external AI is used, what information it relies on and how its performance is assessed. They should also consider dependencies, concentration risks, supplier assurances and fallback or exit arrangements.
The need for visibility will increase as AI becomes more autonomous. The FCA is already considering “agentic finance”, where AI could research products, manage investments and payments, or execute transactions. As systems take on more complex activities, firms will need controls that can track decisions, delegated authority, human intervention and outcomes.
What could compliance teams do now?
Firms should begin by establishing where AI is already being used, including through approved platforms, embedded functionality and less formal employee use.
Practical priorities include:
- maintaining an inventory of AI tools and material use cases
- defining ownership, risk classifications and escalation thresholds
- identifying the customer interactions, documents and decisions affected
- monitoring outputs for accuracy, bias and inconsistent customer outcomes
- ensuring reviewers can access the evidence needed to challenge outputs
- keeping reliable records of testing, decisions, changes and interventions
- reviewing third-party dependencies and resilience arrangements
- using management information to identify patterns and emerging risks
The FCA’s position reinforces the need for proportionate oversight, with clear evidence that firms can readily access. Using AI does not, in itself, mean a firm needs elaborate controls.
Frequently asked questions
No. The FCA’s examples do not create new requirements or prescribe one governance model. Firms must continue to meet existing requirements, including the Consumer Duty, SYSC, SM&CR and operational resilience requirements, when they use AI.
The appropriate evidence will depend on the use case and its level of risk. It could include approvals, testing results, system changes, AI outputs, human interventions, exceptions, customer outcomes and remedial action.
Manual sampling may remain part of a proportionate control framework, particularly for smaller firms, but it can provide a limited view. Broader technology-enabled monitoring can help firms identify patterns, exceptions and risks across a larger proportion of customer interactions and documents.
The regulated firm remains responsible for its obligations and customer outcomes. Firms should understand their reliance on third-party technology and maintain appropriate oversight, assurance, resilience and exit arrangements.