AI agents behaving badly

What institutions must know and be able to prove should an agent make independent decisions

Business units have been asking frequently whether artificial intelligence agents could take over certain payment processes. Bulk payments, for example, have long been automated. Where previously a rule applied or the case was forwarded to a human, an agent can now decide on its own – to hold, approve, request additional information or escalate payment instructions.

While an agent for these tasks can be built without much difficulty, putting one into a production payment process raises different questions of identity, entitlements and control from those in a rule-based process. Where an agent takes on that role, what it is permitted to do and who has authorised it to do so have to be documented from the outset. Under the European Banking Authority guidelines on internal governance, the business lines are responsible for managing the risks they incur in the course of their activities. The use of an agent does not relieve them of this responsibility.

For an individual payment, the institution can still examine the agent’s decision on its own. The difficulty starts when an agent makes hundreds of these decisions during a payment run. If a possible breach of the agent’s mandate only comes to light afterwards, the review starts with the individual payments and works backwards from there. The institution has to understand not only why the agent acted as it did, but whether it was still acting within the authority it had been given.

While the International Monetary Fund note published in April 2026, ‘How agentic AI will reshape payments’, does not set out supervisory requirements, it describes how an agent may act on a mandate granted earlier rather than on an instruction issued for the individual transaction. If an agent makes a payment under an existing mandate, the institution needs to be able to show that the payment fell within that mandate. The speed gained in operation is paid back in the investigation – unless there is a record of what the agent was authorised to do.

Identification and authorisation

In a concept paper published in February 2026, the National Institute of Standards and Technology’s National Cybersecurity Center of Excellence addressed the question of how software and AI agents should be identified and authorised. It asked how an agent’s actions can be audited and made non-repudiable, so that they can be traced back to the authority that originally granted the permissions and it can be determined on what basis the agent acted.

While the institution is trying to establish why the agent acted as it did, the same behaviour may continue across further payments. If it cannot be established quickly whether the agent is still acting within its mandate, the institution may have to suspend the agent or the affected process.

An intervention in the production environment will trigger a series of events. If this prevents the institution from processing specific payments, the disruption may meet the classification criteria for a major ‘ICT-related incident’ under the Digital Operational Resilience Act and become reportable to the supervisory authorities.

Before the agent goes live, the model will usually have been approved and the main risks and operating conditions written down. Even then, the paperwork may not show who allowed the agent to stop a payment, or when a person has to take over again. It would be advisable to maintain a record of authorisations for the agent, clearly stating who granted the authorisation and when the agent must return the decision to a person. The record could form part of the documentation already in place, together with the exit strategy and resilience plan where relevant. This would make it easier to trace the authority given to the agent back to the controls already in place.

Where delegation meets its limits

Not every decision made by an agent has equal importance. An agent may request further information about a payment or send a case for further review. That is very different from stopping a customer from accessing their money or otherwise affecting their rights.

Once a decision can affect a customer’s access to money, staying within the agent’s mandate is only part of the issue. Equally important is whether such a decision may be made entirely without human intervention at all, and when a human must necessarily take over. This is where data protection law also draws a line, with decisions that are made solely by automated means and have legal or similarly significant consequences for an individual being subject to specific restrictions.

Much of the legal framework needed to deal with these questions is already in place. DORA deals with how institutions govern ICT risk, respond to incidents, recover their services and manage dependencies on third parties. The European Union’s second Payment Services Directive deals with the authorisation of payments, strong customer authentication and what happens when a payment was not authorised. The Settlement Finality Directive matters because there comes a point in the payment process when an order can no longer be revoked. Exactly when that happens depends on the rules of the system concerned.

Taken together, all these rules and regulations already cover much of what is relevant here. What they do not necessarily provide, however, is an overview of how the authority granted to the agent fits within this framework.

What the board needs to know

At board level, questions may arise regarding an agent’s powers. For example, which agents are authorised to withhold, authorise or escalate payments and who granted them these powers?

The board must know when the intervention process was last tested under real-world conditions and what the results of that test were. If an agent acts beyond the scope of its mandate and makes independent decisions, but no financial loss is incurred and no key performance indicators are adversely affected or thresholds exceeded, the case may not even be reported to the executive board in the first place.

The payment itself may be correct, even though the agent had no authority to make one of the decisions that led to it. If this happens only once or twice, the institution can still go back and check what happened. However, with hundreds or thousands of decisions, it becomes much more difficult. The institution must then be able to prove who issued the mandate, what it covered and when the decision should have been referred back to a human being.

If these records are incomplete or missing, the institution has to reconstruct the case afterwards, payment by payment, and show what authority the agent had at the time. If that authority can no longer be reconstructed, what remains is a payment that was executed correctly and a decision nobody can account for.

Reza Monshizadeh is a senior technology executive.

Interested in this topic? Subscribe to OMFIF’s newsletter for more.

Join Today

Connect with our membership team

Scroll to Top