marcotcr / lime

Lime: Explaining the predictions of any machine learning classifier
BSD 2-Clause "Simplified" License
11.64k stars 1.81k forks source link

background/filter dataset in LIME #704

Open BinDong314 opened 2 years ago

BinDong314 commented 2 years ago

Hi LIME community, Thanks for the help or any lead in advance.

One thing we are looking for is to provide a background data for LIME at the time of explanation of every instance. We are not sure if it is available in LIME? More specifically, it looks like the shap.KernelExplainer' data parameter. https://shap-lrjball.readthedocs.io/en/latest/generated/shap.KernelExplainer.html

One possible method is to provide feature_selection (e.g, highest_weights) for LimeTabularExplainer or explain_instance_with_data. But, it looks like LimeTabularExplainer needs to build the explainer for every instance, and explain_instance_with_data needs users to provide neighborhood_data etc.

Bests, Bin

BinDong314 commented 2 years ago

Created a pull request here to address my issue. Please help to review it.

https://github.com/marcotcr/lime/pull/705

Some note:

Add predict_fn_accept_dense_only to explain_instance.

When data_row (of explain_instance) is scipy.sparse.matrix but the model was NOT trained with scipy.sparse.matrix, the predict_fn_accept_dense_only convert inverse to dense before call predict_fn (and then back to sparse after call predict_fn)

Bin dbin@lbl.gov