FINALYSE CREDIT RISK MODEL VALIDATION SUITE
Finalyse developed a validation toolkit inspired by the need of credit institutions to automatically produce the ECB supplementary reporting regarding the validation of their IRB models. This toolkit helps banks to build their validation sample for each credit risk parameter (PD, LGD, LGD in-default, EL BE, CCF) and perform all necessary qualitative and statistical tests as defined in these ECB reporting requirements. The toolkit also generates automatic validation reports in the format, structure and labelling consistent with ECB templates. Furthermore, the toolkit is prepared in different programming languages (SAS, R or Python) and packaged in a way that can be easily incorporated into any IT infrastructure.
How does Finalyse address your challenges?
Produce your supplementary validation reports in line with ECB instructions for validation in a simple manner.
Create model validation samples that are aligned with ECB requirements and that include built-in dictionaries.
Easily expand and adjust the suite with additional test metrics in accordance with any institution’s preferences to further automatise the internal validation process for any programming language.
Benefit from 3 different programming languages offered in the suite (SAS, R and Python) which allows it to be adapted without any change to the IT infrastructure.
Automatically report the test results and qualitative portfolio information in the required format from the ECB reporting template for supplementary validation reports.
Calculate a wide-range of quantitative and qualitative test metrics for any IRB risk parameter in line with the ECB instructions and internal validation standards.
How does it work in practice?
In line with ECB instructions, validation test metrics are defined for following categories in each risk parameter:
- Qualitative tests: Summary statistics for IRB parameters and problematic data entries (outliers, technical defaults, etc.)
- Predictive ability: Assessment of the calibration accuracy of the model by comparing the estimated and realized values [Further potential test metrics: Vasicek test, Binomial test]
- Discriminatory power: Analysis of the rank-ordering ability of the model [Further potential test metrics: KS-statistic, Spearman’s rank correlation]
- Stability: Comparison of the risk characteristics of the portfolio (i.e. model application scope) at the beginning and end of a given observation period
- Concentration: Assessment of variation in risk parameter estimates
Finalyse validation toolkit can also be easily adjusted and expanded to include additional metrics to comply with institutions’ internal validation standards if required. The validation toolkit already contains some additional metrics that are commonly used in banking industry covering additional test categories such as homogeneity, heterogeneity or sensitivity. These test metrics can be used for internal validation reports, although they are not mentioned in ECB reporting instructions. In general, internal validation reports for all IRB risk parameters can also be automatized in a similar way by aligning the Finalyse validation toolkit with internal validation standards.
Key Features
The toolkit is to be used in the validation process of the IRB models:
- Regular monitoring of the existing models in line with ECB requirements,
- Supplementary ECB reporting tool for supervised banks,
- Automation of the internal validation process and internal validation reporting.
What Finalyse experts say
The three languages were compared using a simple setup, as close as possible to a real-life situation. The exercise consisted in calibrating a logistic regression to identify loans likely not to be repaid on time in a sample dataset. The choice of logistic regression was driven by the fact that it is a simple but powerful approach still widely used in the industry.
ReadThis article introduces the challenges of the ECB supervised entities regarding their internal validation of the IRB Approach. It also considers a benefit of use of Model Validation tool.
Read