search

ModelOriented / DALEX

moDel Agnostic Language for Exploration and eXplanation
https://dalex.drwhy.ai
GNU General Public License v3.0
1.38k stars 166 forks source link

How to interpret `baseline` in `variable_importance()` plots? #42

Closed kevinykuo closed 6 years ago

kevinykuo commented 6 years ago

How do we interpret the baseline in variable importance? Noticed that when the variable importances for different models are plotted simultaneously that the baseline numbers don't agree, even with type = "ratio", n_sample = -1.

kevinykuo commented 6 years ago

@pbiecek looking for your input on this, will be talking about DALEX applications in upcoming talks so I want to keep myself honest 😄

mstaniak commented 6 years ago

Baseline is the change in model performance when ALL variables are permuted, so it depends on the model. Does this help?

kevinykuo commented 6 years ago

Makes sense, thanks!

pbiecek commented 6 years ago

@kevinykuo one way to think about baseline is that it is an average model response. And as mstaniak already wrote, different models may have different average responses.