Running the California Housing Dataset in the low code environment to understand each of the different models, comparing test MSE and train MSE
If test MSE is much worse than train MSE, the model isn’t a good fit, i.e., want the difference between the train MSE and test MSE to be small
For comparison purposes, starting with the EBM model it is much easier in terms of explainability .
If the value of each attribute/ variable within a dataset is known the effect of each observation (prediction) can be manually computed. Eg: if we know longitude is less than 0.6 we can compute the effect to be less than 0.1. 0.6 and 0.1 can be found on plots showing interaction and sum them to find the interaction effect (research more for greater clarity).
The effect plots are the best choice when drawing comparisons between the explainable/interpretable models and the black box model (Look further into how the feature plots translate to interaction plots, and compare with PDPs).
Focus on EBM versus Black Box Model rather than comparing various models, once understood, move on to other models versus black box models.
Look into model robustness through plots in the low code environment (compare with two models, under model comparison and benchmarking section). Use the quantile scale under noise scale. (AUC, Area under the ROC (receiver operating characteristic curve) curve) curve, will be seen for classification models).
Meeting Minutes (03/06/23)