Kim XNN TODOs

[jphall663]

Features

• [x] Input feature list (hopefully informed by @navdeep-G using GBM Shapley)

For simulated data

• [x] Unconstrained feedforward ANN trained w/ 5-fold CV with training/CV and test AUC, Accuracy, RMSE, logloss
• [x] XNN trained w/ 5-fold CV with training/CV and test AUC, Accuracy, RMSE, logloss
• [x] Mean local feature importance values across quintiles of predictions (by Shapley or gradient-based) for XNN for top 5 features
• [x] Ridge function curves for XNN for top 5 features
• [x] ICE curves at quintiles of predictions for XNN for top 5 features

For mortgage data

• [x] Unconstrained feedforward ANN trained w/ 5-fold CV with training/CV and test AUC, Accuracy, RMSE
• [x] XNN trained w/ 5-fold CV with training/CV and test AUC, Accuracy, RMSE
• [x] Mean local feature importance values across quintiles of predictions (by Shapley or gradient-based) for XNN for top 5 features
• [x] Ridge function curves for XNN for top 5 features
• [x] ICE curves at quintiles of predictions for XNN for top 5 features

Fairness

• [x] Pandas frame of predictions and row IDs for the test data for @nickpschmidt to conduct discrimination testing for XNN

[jphall663]

from 12/6 meeting: (supercedes above)

• [x] test accuracy, auc, logloss, RMSE
• [ ] try for shapley values in the margin\logodds space: model_output=margin?
• [x] provide csv of shapley values, preferably in both spaces, margin and prob (patrick will make global shap importance and local shap plot)
• [x] provide CSV of ridge functions for top 3 features (patrick will make histograms)
• [x] provide frame of probs for nick