Week 3/6

[reinabhatkuly]

Meeting Minutes (03/06/23)

• Running the California Housing Dataset in the low code environment to understand each of the different models, comparing test MSE and train MSE
• If test MSE is much worse than train MSE, the model isn’t a good fit, i.e., want the difference between the train MSE and test MSE to be small
• For comparison purposes, starting with the EBM model it is much easier in terms of explainability .
• If the value of each attribute/ variable within a dataset is known the effect of each observation (prediction) can be manually computed. Eg: if we know longitude is less than 0.6 we can compute the effect to be less than 0.1. 0.6 and 0.1 can be found on plots showing interaction and sum them to find the interaction effect (research more for greater clarity).
• The effect plots are the best choice when drawing comparisons between the explainable/interpretable models and the black box model (Look further into how the feature plots translate to interaction plots, and compare with PDPs).
• Focus on EBM versus Black Box Model rather than comparing various models, once understood, move on to other models versus black box models.
• Look into model robustness through plots in the low code environment (compare with two models, under model comparison and benchmarking section). Use the quantile scale under noise scale. (AUC, Area under the ROC (receiver operating characteristic curve) curve) curve, will be seen for classification models).

[agallagher23]

Tasks completed

• Preliminary research on ReLU-DNN model
  • Watched videos and read articles, found it overall pretty confusing/unable to grasp the foundational concepts
• Communicated with group to set agenda for meeting with Nengfeng
• Compiled the results of the EBM model on California housing data with Reina

[xybljy0122]

3/6 zoom meeting recording - model comparison walk through

https://drive.google.com/file/d/1eWqnzB_oXmXtu-9ysj-ln-hD-EqXrNZK/view?usp=sharing

[everettbu]

Tasks Completed

• Looked into GAMI-Net model
• Compared differences in accuracy of the GAMI-Net, EBM and ReLU-DNN models
• Compile the results of the Light-GBM model on California housing data (to do today)

[xybljy0122]

Tasks Completed

• Research on Light-GBM model(blackbox), and run the results through post hoc analysis provided in Pi-ML
• Compile the results of the Light-GBM model on California housing data

[reinabhatkuly]

Tasks Completed

• Research on EBM Models
• Took down meeting minutes & did some research on train and test MSE
• Compiled the results of the EBM model on California housing data with Aly