search

TeamHG-Memex / eli5

A library for debugging/inspecting machine learning classifiers and explaining their predictions
http://eli5.readthedocs.io
MIT License
2.74k stars 332 forks source link

Implementation for multilabel keras model #393

Closed tkmamidi closed 3 years ago

tkmamidi commented 3 years ago

Hello, Is there an implementation for using PermutationImportance for multi class models (using tensorflow sequential)? Please help! Thanks in advance :)

lopuhin commented 3 years ago

My understanding is that PermutationImportance assumes an sklearn model (although it requires very little from it so it may be quite easy to wrap a TF model in a suitable interface), but get_score_importances (https://eli5.readthedocs.io/en/latest/autodocs/permutation_importance.html) is model-agnostic as it only needs a score_func to be defined.

tkmamidi commented 3 years ago

Thanks for the suggestion. I'm also trying to implement "explain_prediction" for my multiclass sequential model. I get the following error when using the following command: eli5.explain_prediction_keras(model,test_x[0]) Error: model "sequential" is not supported, try passing the "image" argument if explaining an image model.

Any help is appreciated. Thanks in advance :)

teabolt commented 3 years ago

Hey. eli5.explain_prediction_keras is only for explaining Keras models using the Grad-CAM method. Currently it only supports image classifiers https://github.com/TeamHG-Memex/eli5/blob/master/eli5/keras/explain_prediction.py#L110.

tkmamidi commented 3 years ago

Is there an implementation to explain predictions for categorical multi-class sequential models? Please help! Thanks in advance!

teabolt commented 3 years ago

Multi-class models are supported, but the input of the model has to be an image. There's also a PR for text inputs https://github.com/TeamHG-Memex/eli5/pull/325. If your model doesn't accept images or text and you still want to use Grad-CAM, I'm not sure if there is an easy way. Grad-CAM only makes sense if your model layers have spatial information, like convolutional layers or recurrent layers do.

I'm not sure if a black box method can be used to explain a single prediction?

tkmamidi commented 3 years ago

SHAP has an implementation but was wondering if eli5 has something similar. https://slundberg.github.io/shap/notebooks/Iris%20classification%20with%20scikit-learn.html