Bank of England warns of AI hacking risks in financial services
The Bank of England has warned financial services firms to brace for cyber-attacks amid a rise in AI test security incidents and threats to the UK’s financial stability.
The risk outlook for the UK economy has worsened since July, while AI security incidents put pressure on firms to prepare for AI-related cyber and operational risks, according to the Bank’s Financial Policy Committee (FPC).
The Bank of England’s latest meeting put a spotlight on the rate at which AI technology is evolving and the subsequent risks that follow.
Simon Edwards, CEO of SE Labs, commented: “The Bank of England is absolutely right to put the microscope on testing for AI risks. Any AI tools within a bank or financial firm need to undergo stringent testing like any other security product, especially considering the vast amounts of confidential data that can be accessed.”
“This involves independent testing using real-world attack techniques used by hackers before any deployment goes live. The same applies to the security systems being used to defend against AI threats. Security teams need to have confidence that their defences will work against these threats, and they can’t just take a vendor’s word for it.”
Andrew Bailey, the Bank’s governor, said separately that AI advances can’t be viewed in isolation to the UK’s financial system.
“The capabilities of frontier models are advancing quickly and our understanding of them needs to keep pace, especially as they are put to use outside controlled test environments,” Bailey said.
Dr Janet Bastiman, chief data scientist at Napier AI, commented: “The real concern is not that AI can now perform sophisticated tasks autonomously, but that we are still learning how to reliably test and predict what happens when those capabilities are deployed in the real world. A model performing well in a controlled test is not evidence that it will behave reliably under different conditions.”
“As AI systems become more autonomous, evaluation needs to move beyond static benchmarks towards continuous testing of behaviour, including how models respond to unexpected inputs, interact with other systems and fail. The question is no longer simply what AI can do. It is whether we can demonstrate, with sufficient evidence, when it can be trusted to do it.”
It comes following a major rogue agent incident in July, where OpenAI’s models autonomously hacked into AI company Hugging Face during sandbox testing. The models escaped the sandbox and used stolen login credentials and a previously unknown security flaw to access Hugging Face’s servers.