Citi has used AI and machine learning tools for many years and has more recently begun to broadly deploy Generative AI, including within its technology platforms and services. In the future, Citi expects to more broadly integrate Generative AI tools within its systems, businesses and functions, including advanced AI capabilities, such as autonomous agents and sophisticated user interactions, which if improperly managed, could result in increased risks and costs.
While Citi has policies that govern the use of emerging technologies, including in model risk management, ineffective, inadequate or faulty Generative AI development or deployment practices by Citi or third parties, including insufficient testing and monitoring, poorly structured or manipulated prompts or insufficient or inadequate human oversight, could result in adverse consequences, such as AI algorithms that produce inaccurate or incomplete output or output based on biased, incomplete and/or inaccurate datasets, infringe on intellectual property rights of others, involve data exfiltration risks, including release of confidential or proprietary information, or cause other issues, concerns or deficiencies. The complexity of AI models, particularly Generative AI models, can make it challenging to understand why particular outputs are generated, which can increase the risk of erroneous and/or biased output and complying with regulations requiring documentation, explainability or auditability on the basis of AI-influenced decisions.
Citi may also rely on AI models developed by third parties and be exposed to risks arising from their development and training methods, including potential inclusion of unauthorized material in the training data or limitations in their risk mitigation strategies, over which Citi may have limited visibility or control (see also the operational processes and systems risk factor below). While Citi may provide restrictions on use of certain data in third-party AI applications or models, a third party could breach those terms, which could expose Citi to legal and reputations risks. Citi is also exposed to risks related to the use of AI technologies by counterparties, clients and vendors, where interconnected AI systems could amplify failures and threats of cyber infiltration, as well as cause widespread disruptions.
Moreover, the use of increasingly sophisticated AI technologies by malicious actors and others has increased the risk of fraud, including identity theft and bypassing of verification controls, and exposure to cyberattacks (see the cybersecurity risk factor below), as well as disinformation and market manipulation campaigns, and failure to effectively manage such risks could result in misappropriation of funds, unauthorized transactions, exposure of sensitive client or Company information, reputational harm and increased litigation and regulatory risk.
Citi also faces competition risks to the extent that competitors may be faster and more successful in developing and deploying AI technologies to improve processes, productivity, efficiency, products and services, and thereby gain competitive advantages over Citi (see the competition risk factor above).
In addition, compliance with new or changing laws, regulations or industry standards relating to AI may impose additional operational risks and costs. Failure to sufficiently manage these risks could expose Citi to adverse legal, regulatory or reputational consequences.
While Citi has policies that govern the use of emerging technologies, including in model risk management, ineffective, inadequate or faulty Generative AI development or deployment practices by Citi or third parties, including insufficient testing and monitoring, poorly structured or manipulated prompts or insufficient or inadequate human oversight, could result in adverse consequences, such as AI algorithms that produce inaccurate or incomplete output or output based on biased, incomplete and/or inaccurate datasets, infringe on intellectual property rights of others, involve data exfiltration risks, including release of confidential or proprietary information, or cause other issues, concerns or deficiencies. The complexity of AI models, particularly Generative AI models, can make it challenging to understand why particular outputs are generated, which can increase the risk of erroneous and/or biased output and complying with regulations requiring documentation, explainability or auditability on the basis of AI-influenced decisions.
Citi may also rely on AI models developed by third parties and be exposed to risks arising from their development and training methods, including potential inclusion of unauthorized material in the training data or limitations in their risk mitigation strategies, over which Citi may have limited visibility or control (see also the operational processes and systems risk factor below). While Citi may provide restrictions on use of certain data in third-party AI applications or models, a third party could breach those terms, which could expose Citi to legal and reputations risks. Citi is also exposed to risks related to the use of AI technologies by counterparties, clients and vendors, where interconnected AI systems could amplify failures and threats of cyber infiltration, as well as cause widespread disruptions.
Moreover, the use of increasingly sophisticated AI technologies by malicious actors and others has increased the risk of fraud, including identity theft and bypassing of verification controls, and exposure to cyberattacks (see the cybersecurity risk factor below), as well as disinformation and market manipulation campaigns, and failure to effectively manage such risks could result in misappropriation of funds, unauthorized transactions, exposure of sensitive client or Company information, reputational harm and increased litigation and regulatory risk.
Citi also faces competition risks to the extent that competitors may be faster and more successful in developing and deploying AI technologies to improve processes, productivity, efficiency, products and services, and thereby gain competitive advantages over Citi (see the competition risk factor above).
In addition, compliance with new or changing laws, regulations or industry standards relating to AI may impose additional operational risks and costs. Failure to sufficiently manage these risks could expose Citi to adverse legal, regulatory or reputational consequences.