search

suinleelab / treeexplainer-study

Code and documentation for experiments in the TreeExplainer paper
178 stars 44 forks source link

Model explanation without any base value? #1

Closed wptmdoorn closed 4 years ago

wptmdoorn commented 4 years ago

Dear authors,

First of all - congratulations on the amazing publication in Nature Machine Intelligence. Absolutely well deserved and the paper itself is full of amazing data and results.

Recently I have been investing a lot of time into potentially integrating SHAP into machine-learning based clinical decision support systems. Hereby we would like to provide the user (a clinician and/or nurse) a brief summary of why the algorithm provided us with e.g. a certain risk score. We thought, and talked a lot about how we should implement this; ultimately aiming to provide an interface which a clinician can actually use.

Currently our code mainly is driven by TreeExplainer methods combined with creating a customized force-plot where we set it's base value to the population of healthy "baseline" individuals. For example, using this approach, we can show the clinician the most important features that actually contribute to a higher risk of a specific individual.

With interest I read your paper and I was mainly intrigued by the figure below. Score

Here you provide a 'white box' local explanation which directly corresponds to the mortality risk (probability) of the model. I think this is a very, very clear explanation of how the model came to its explanation. I am just not sure how you achieved this. As far as I know, we always want to define a base value - but that seems to be missing here? Did you correct afterwards? Could you kindly elaborate on this figure?

Thanks a lot in advance.

slundberg commented 4 years ago

Great question! That figure treats the left side of the box as the base value, so really it is implicitly hiding it. When I worked on extending that figure into the code I decided to explicitly show the base value as can now be seen in the updated version of that figure at: https://github.com/slundberg/shap/blob/master/README.md

If you want to make your own version give the new shap.waterfall_plot function a try :)

wptmdoorn commented 4 years ago

Woah, I was not aware of the shap.waterfall_plot function at all. Guess this shows why I should keep up to date with packages.. :-) (I still was on 0.27).

I will give it a shot. I also like your extended figure, but intuitively I feel like the base-value concept is sometimes hard to explain to layman users. Again, thanks for your reply on such a short notice and even more so for creating such a wonderful package :+1:. I will close the issue for now.