Practice area
Model validation.
Model validation that satisfies regulatory requirements and delivers more than a passing opinion. Technical findings on model performance and the development lifecycle — giving the second line of defense something to build on.
Independent validation, by someone who has built models.
A validation report should hold up to an examiner — and to the modeler whose model it covers.
The SR 11-7 framework asks for independent validation across three dimensions: conceptual soundness, ongoing monitoring, and outcomes analysis. Done well, a validation gives the model risk committee real comfort. Done poorly, it gives them a 60-page document that says 'no material issues identified' and an examiner who finds three.
Our practice covers credit risk models (PD/LGD/EAD, CECL, CCAR loss forecasting), market risk models (VaR, sensitivity, stress), AML transaction monitoring models (rule sets and machine-learning hybrids), and the increasing class of AI/ML models used in underwriting, fraud detection, and customer-facing applications. For each, we validate against the data, the assumptions, the implementation, and the use.
The validator has to understand the model. That sounds obvious; it is what most validation reports fail at. Edgar built his career as a quantitative practitioner before he became a validator. The reports our practice produces show the math; they do not paper over it.
The work in this practice, named.
- Credit risk models — PD/LGD/EAD, CECL allowance models, CCAR/DFAST loss forecasting, scorecard models.
- Market risk models — VaR, expected shortfall, sensitivity, scenario and reverse-stress models.
- AML / TM models — Rule-set calibration, threshold tuning, hybrid ML-based detection systems.
- AI / ML models — Underwriting, fraud, churn, and customer-facing models — including fairness, explainability, and drift monitoring.
- Conceptual soundness — Theory, assumptions, choice of methodology, alternatives considered, data appropriateness.
- Ongoing monitoring & outcomes — Backtesting, benchmarking, sensitivity, monitoring plan, threshold setting.
A model validation, beginning to end.
| Phase | Deliverable |
|---|---|
| Intake | Model documentation reviewed, data dictionary received, scope confirmed with model risk management. |
| Replication | Independent replication on the same data; alternative specifications considered. |
| Testing | Sensitivity, stability, fairness (where applicable), backtesting, benchmarking. |
| Reporting | Findings rated, validation report drafted, MRMC presentation prepared. |