Our goal is to mutually develop a governance framework that manages the on-boarding, development of, and running AI-based solutions within financial services organisations - allowing us all to unlock the potential of this disruptive technology in a safe, trustworthy and compliant way.
Problem statement: As organizations rollout third party services underpinned by Large Language Models they are subject to model validation requirements imposed by the OCC related to third party model risk management. In particular, when adopting SaaS services with rapidly evolving frontier Models, MRM model quality/security validation processes can find themselves falling behind with a laborious validation process. There is a need for a continuous validation process that allows organizations to build and maintain, in real time, trust in model quality and security.
Latest Thinking: The organization considers layers of Model Validation from the creator of the frontier model (ex. OpenAI), the SaaS provider and a self managed continuous monitoring process.
Problem statement: As organizations rollout third party services underpinned by Large Language Models they are subject to model validation requirements imposed by the OCC related to third party model risk management. In particular, when adopting SaaS services with rapidly evolving frontier Models, MRM model quality/security validation processes can find themselves falling behind with a laborious validation process. There is a need for a continuous validation process that allows organizations to build and maintain, in real time, trust in model quality and security.
Latest Thinking: The organization considers layers of Model Validation from the creator of the frontier model (ex. OpenAI), the SaaS provider and a self managed continuous monitoring process.
Resources:
OCC Requirements
Open Needs: