In my PhD, I developed a method for tuning model complexity and controlling overfitting in ML. It can be used to evaluate when the ML is using the data structure to explain that dataset or sample (overfitted) rather than the underlying phenomena, and which level of model complexity that dataset can allow (based on the number of predictors in relation to observations, covariance, distribution of the response variable).
The methodology can be adapted to the concept of interpretable ML, as the method help to understand what is the model contribution to explain the response variable before it overfits by the excess of model complexity.
If you are interested, I can explain better the simple concept behind it, and help to adapt it to be part of the "agostic methods". The application is in remote sensing, but I am a statistician, so I see the potential to use it in other areas, especially in cases when there is a big quantity of predictors and limited observations to support it,
Hi Christoph,
In my PhD, I developed a method for tuning model complexity and controlling overfitting in ML. It can be used to evaluate when the ML is using the data structure to explain that dataset or sample (overfitted) rather than the underlying phenomena, and which level of model complexity that dataset can allow (based on the number of predictors in relation to observations, covariance, distribution of the response variable).
Please see attached Rocha_et_al_2017-ISPRS.pdf , "The Naïve Overfitting Index Selection (NOIS): A new method to optimize model complexity for hyperspectral data"(https://doi.org/10.1016/j.isprsjprs.2017.09.012)
The methodology can be adapted to the concept of interpretable ML, as the method help to understand what is the model contribution to explain the response variable before it overfits by the excess of model complexity.
If you are interested, I can explain better the simple concept behind it, and help to adapt it to be part of the "agostic methods". The application is in remote sensing, but I am a statistician, so I see the potential to use it in other areas, especially in cases when there is a big quantity of predictors and limited observations to support it,
Thank you, Alby Rocha_et_al_2017-ISPRS.pdf