8
Responsible AI prompts a renewed focus on risk

Working closely with risk teams and refreshing processes will help banks deploy AI safely and build trust.

Responsible AI prompts a renewed focus on risk

Banks are alert to the risks of rapidly scaling AI. The quality of the output of AI-powered tools is their main concern, especially if they provide advice directly to clients. Consider the reputational damage of an AI chatbot advising a client to transfer money in a way that breaches regulations, for instance.

The EY Responsible AI Pulse survey found that two-thirds of banking leaders consider unreliable AI output as a major to moderate concern, and 48% worry about false AI-generated information being taken seriously.

They also worry about data, and that personal or proprietary data could be compromised when it’s used in AI models. “Data leakage is one of the greatest risks for us,” explains MUFG Bank’s Abe. “We might not want our proprietary data to be pooled with that of other companies in LLMs — we don’t know what we’re mixing it with.

We’d value being able to curate the set of data specifically for our business and for our needs.”

Data leakage is one of the greatest risks for us. We might not want our proprietary data to be pooled with that of other companies in LLMs – we don’t know what we’re mixing it with. We’d value being able to curate the set of data specifically for our business and for our needs.

Osamu Abe, Chief of Staff for Asia-Pacific, MUFG Bank

Based on the EY Responsible
AI Pulse survey

consider unreliable AI output to be a major to moderate concern

worry about false AI-generated information being taken seriously

Source: EY Responsible AI Pulse survey.

Risk management is about to get more complicated

How can banks mitigate these threats? As a starting point, they’ll need a more sophisticated approach to model risk management and model governance for the complex, language-based GenAI models. And that means new skills.

Second, they’ll need to address concerns about transparency at the outset of designing an AI application and the surrounding workflows. For example, designers working on knowledge management tools could specify that any advice created by LLMs must reference source material. A thorough risk assessment should also determine when human involvement is needed.

In parallel, technology teams should work alongside the risk function to identify the sequence of checks that need to be considered before a GenAI use case is scaled. ING has done exactly this. “GenAI brings a lot more risk dimensions relating to privacy, IP [intellectual property], vendors, data and decisioning, so you need to find a way of using the technology safely,” says ING’s Yilmaz.

“We have 140 checks that need to be assessed for GenAI, though some of these are being automated.”

Other strategies are available to banks that want to manage risk and build trust in their use of AI. The EY Responsible AI Pulse survey found that just 58% of banks use third-party attestations or internal audits to provide confidence in their AI systems, while 52% have documented AI processes, established an ethics policy or undertaken a regulatory compliance assessment. There’s significant scope for more corporate and commercial banks to adopt this approach.

GenAI brings a lot more risk dimensions relating to privacy, IP [intellectual property], vendors, data and decisioning, so you need to find a way of using the technology safely. We have 140 checks that need to be assessed for GenAI, though some of these are being automated.

Bahadir Yilmaz, Chief Analytics Officer, ING

Key actions

Refresh identification and mitigation of risks as more GenAI models are deployed.
Involve risk teams at the outset of major AI use cases to ensure threats are considered at an early stage.
Update model risk management processes to reflect the growing complexity of GenAI.