Navdeep MGBM TODOs

[jphall663]

Features

• [x] Input feature list
• [x] Global feature importance rankings for @kmontgom2400

For mortgage data

• [x] Modelling:
  • [x] Unconstrained GBM trained w/ random grid search and 5-fold CV with training/CV and test AUC, Accuracy, RMSE
  • [x] constrained MGBM trained w/ random grid search and 5-fold CV with training/CV and test AUC, Accuracy, RMSE
• [x] MLI:
  • [x] Mean local feature importance values across quantiles of predictions (by Shapley) for MGBM for top 3 features
  • [x] Partial dependence curves for MGBM for top 3 features
  • [x] ICE curves at quantiles of predictions for MGBM for top 3 features

For simulated data

• [x] Modelling:
  • [x] Unconstrained GBM trained w/ random grid search and 5-fold CV with training/CV and test AUC, Accuracy, RMSE
  • [x] constrained MGBM trained w/ random grid search and 5-fold CV with training/CV and test AUC, Accuracy, RMSE
• [x] MLI:
  • [x] Mean local feature importance values across quantiles of predictions (by Shapley) for MGBM for top 3 features
  • [x] Partial dependence curves for MGBM for top 3 features
  • [x] ICE curves at quantiles of predictions for MGBM for top 3 features

Fairness

• [x] Pandas frame of predictions and row IDs for the test data for @nickpschmidt to conduct discrimination testing for MGBM
  • [x] Models are saved under /models. Just need to use the .predict() function on data to get predictions.

[jphall663]

from 12/6 meeting: (supercedes above)

• [x] test accuracy, auc, logloss, RMSE
• [x] try for shapley values in the prob space: tree explainer on booster?
• [x] provide csv of shapley values, preferably in both spaces, margin and prob (patrick will make global shap importance and local shap plot)
• [x] provide CSV of PDP and ICE for top 3 features (patrick will make histograms)
• [x] provide frame of probs for nick

[navdeep-G]

I think I have everything from this issue in the repo @jphall663.