search

suinleelab / DeepSHAP

Experiments for DeepSHAP paper.
14 stars 2 forks source link

shap values dimension and per_reference setting #1

Open DonCorle0ne opened 1 year ago

DonCorle0ne commented 1 year ago

Hello,

Thanks for providing this great content.

I am trying replicate the Lending example notebook, but running into some issues. Here are some questions I have:

  1. The dimension of SHAP value that I got maybe a bit different from yours, for example:

explainer = shap.TreeExplainer(fraud_mod, refe_frau, feature_dependence="interventional") attr_frau = np.swapaxes(explainer.shap_values(expl_frau, per_reference=True),1,2) explainer = shap.DeepExplainer(credit_mod, refe_cred.values) attr_cred = explainer.shap_values(expl_cred.values, per_reference=True)[0] explainer = shap.TreeExplainer(bank_mod, refe_bank, feature_dependence="interventional") attr_bank = np.swapaxes(explainer.shap_values(expl_bank, per_reference=True),1,2)

expl_frau and refe_frau are the same dimensions as the data, 1000 3 and 100 3 correspondingly. The output of explainer.shap_values(expl_frau, per_reference=True) should be 1000*3, since there is a shap value for each record and each feature. However, the np.swapaxes(, 1,2) here would not work since I do not have a third dimension.

  1. I ran into error saying TypeError: shap_values() got an unexpected keyword argument 'per_reference'. What is this used for?
HughChen commented 1 year ago

This is based on a pull request that never got merged in the shap repo unfortunately. If you are interested to try it out, it should be available here: https://github.com/slundberg/shap/pull/346. I think in that PR the parameter which corresponds to per_reference is model_stack.

DonCorle0ne commented 1 year ago

@HughChen Thanks for replying. I assume the per_reference parameter here will generate a SHAP matrix for each individual reference record. Say if expl_frau is 1000 10, refe_frau is 100 10, then explainer.shap_values(expl_frau, per_reference=True) will give a result of dimension 100 1000 10. Let me know if it makes sense.