Model risk management is the discipline of identifying, measuring, and controlling the risk that a model produces wrong outputs or is used in the wrong way, and that the organization suffers financial, regulatory, or reputational harm as a result. It is decades old in banking and insurance, where a mispriced risk model can cost hundreds of millions. What is new is the range of models in scope: alongside the credit scorecard and the actuarial model now sit foundation models, large language models, and AI agents that generate text, extract data, and increasingly take actions.
For regulated US firms, this is not optional hygiene. It is supervisory expectation. Guidance such as the Federal Reserve and OCC bulletin SR 11-7 has long defined the standard: models are subject to independent validation, effective challenge, documentation, and ongoing monitoring, with clear accountability for each one. When a bank or insurer deploys an LLM, examiners apply that same lens. The question is never whether the model is impressive. It is whether you can show the model is fit for its purpose and stays that way.
The three pillars of model risk
Sound model risk management rests on three well established pillars, and they map cleanly onto AI.
Independent validation. Someone other than the model’s builders confirms it works: the data is appropriate, the methodology is sound, the performance holds on realistic inputs, and the limitations are documented. Independence is the point. A team validating its own model tends to grade generously.
Effective challenge. A credible reviewer with the standing and the incentive to say no probes the assumptions and pushes back before the model reaches production. Challenge is where weak models get caught.
Ongoing monitoring. A model that passed validation at launch can degrade as the world shifts. Monitoring watches performance over time and triggers revalidation when it slips.
Why AI stresses the traditional playbook
Foundation models and LLMs break several assumptions the classic playbook relied on. A traditional model is often transparent enough to inspect equation by equation. A large language model is not, so validation leans harder on behavioral testing: how does it perform across representative and adversarial inputs, where does it fail, and how badly. AI also introduces failure modes the old models did not have.
Hallucination. The model states something false with total confidence. In a regulated decision, a confident wrong answer is worse than a visible gap.
Drift. Model behavior, input data, and the surrounding environment change over time, quietly eroding performance until someone notices the outcomes have moved.
Prompt injection and misuse. Because these systems take instructions in natural language, a hostile input can try to redirect them. That is a security and a model risk problem at once.
Third-party dependence. Most firms consume foundation models from a provider rather than building them. Vendor model risk, understanding what you are relying on and how it changes, becomes a first-class concern.
Because of these, AI validation cannot be a one time gate. It has to be continuous: evaluation baselines, monitoring for drift and hallucination, guardrails against injection, and a documented trail that an examiner or an internal auditor can follow.
What good looks like in production
A defensible program keeps a live inventory of every model with an owner, a risk tier, and a validation status. High-risk use cases, anything touching credit, underwriting, fraud, or customer treatment, get deep validation and a human in the loop before actions land. Every model carries documentation that explains what it does, what data it uses, where it should not be trusted, and how it is monitored. And the whole thing is auditable, so six months after a decision you can reconstruct why the model produced what it did. Running models inside your own cloud, for example your AWS environment, keeps the data and the evidence under your existing controls, which makes that audit trail far easier to stand behind.
Working with BlueMetrics
BlueMetrics brings model risk management to AI in regulated contexts through BlueRisk, our approach to putting validation, monitoring, and documentation around models before they reach production. We help risk and data teams treat foundation models and LLMs with the same rigor as any other model that carries regulatory weight: independent evaluation, continuous monitoring for drift and hallucination, guardrails against misuse, and an audit trail that holds up under examination, all inside your own AWS environment. If you are deploying AI into decisions that a regulator will look at, explore BlueRisk to see how we keep those models defensible in production.