CECL Implementation Tips

Tags
Blog Post
Created by
Marc Leprince
Created time
Feb 6, 2024 2:10 PM
Last edited time
Feb 21, 2024 3:53 PM
Medium Post
Published Date

I was reading an old white paper written by my colleagues back at SAS on CECL Implementation tips. After working through several loss forecasting implementations, it is amazing how some of the statements I once glossed over, now resonate. I wanted to highlight a few that I believe make for the smoothest implementations.

No new implementation occurs without it’s hiccups, but as someone who has relearned many of the lessons taught in this paper, following these guidelines can save a lot of headaches. Here are a few of my favorites:

  • Separating data preparation for scoring data, model estimation & model scoring. Only data prep for scoring data & model scoring need to be incorporated into the production cycle.
    • After seeing this at a number of clients both large & small - I have seen how developers throw their code over the wall to implementers (or even implement it themselves). For various reasons: to make tie-outs easier, quickly answer audit as to why/how the model they developed is “different” than what is in production… For a production process to be smooth and efficient, only the essential parts need to be there.
    • Second, bringing in model estimation is a bad design choice because it brings too much of the volatility in model development into a production process. It compounds the diversity, depth and width of data required, which complicates data preparation flows. Next, this creates unnecessary processing time required to generate model coefficients that would otherwise remain the same for ALL model runs in a particular reporting period (month, quarter, etc…). While often small for each segment, this extra time weighs from development, testing, implementation and time-critical production runs. Once you begin to multiply these problems by portfolio segment and model… This can easily become a significant project of it’s own.
  • Standardizing scoring data & it’s creation process
    • With any regulatory reporting process, there are strict governance and controls to ensure accuracy of model implementation and model results generated. To validate both of these - requires a set of scoring data that will be run in development, testing and production environments. Too often do we see development datasets differ from production and it creates tie-out validation problems that can take hours across multiple teams to resolve. While differences may occur for valid reasons such as exploratory development, institutions should strive to ensure that scoring datasets are the same for development, and testing purposes.
    • Second, a result of exploratory development is the creation of many manual processes to update/alter data through custom transformations. While fine for development, these do not scale and will not be allowed for a production run. These should be automated as part of the production process.
  • Portfolio Segmentation logic & Data Mapping
    • Models in production need to be run multiple times. For scenario analysis, as part of attribution analysis, sensitivity & a variety of other reasons. Setting up a production model run involves advanced data prep, scenario mapping and a host of other work to setup the right model version & other risk data inputs. Without a transparent view into data dependencies, logic flow & output generated, all the work described must be performed for EVERY model run - because it is treated by production as an immutable black box. This can create mountains of duplicate, redundant work for teams often on critical timelines. To simplify this - the details of the data dependencies and segmentation/modeling flow must be surfaced so that end-users have the ability to decide what they want to run and how. Not only is this an enormous value add for finance/risk teams, but it also saves on processing time and provides a level of understanding rather than another black box process.