NorskRegnesentral / shapr

Explaining the output of machine learning models with more accurately estimated Shapley values
https://norskregnesentral.github.io/shapr/
Other
147 stars 34 forks source link

Documentation claims shaprpy can explain Keras models, but code claims it cannot #394

Open mdhimes opened 7 months ago

mdhimes commented 7 months ago

Hello. I am using the Python interface, shaprpy. The documentation for it claims that "shaprpy knows how to explain predictions from models from sklearn, keras and xgboost. For other models, one can provide a custom predict_model function (and optionally a custom get_model_specs) to shaprpy.explain."

However, when using a Keras model, the following warning arises: ".../shapr/python/shaprpy/explain.py:255: UserWarning: No pre-built get_model_specs for model of type <class 'keras.src.engine.functional.Functional'>, disabling checks. warnings.warn(f'No pre-built get_model_specs for model of type {type(model)}, disabling checks.') R[write to console]: Note: You passed a model to explain() which is not natively supported, and did not supply a 'get_model_specs' function to explain(). Consistency checks between model and data is therefore disabled."

This quite plainly states that a Keras model "is not natively supported", which seems at odds with the documentation. Looking in the code seems to confirm this: https://github.com/NorskRegnesentral/shapr/blob/master/python/shaprpy/explain.py#L313 Note that only sklearn and xgboost are included in this function, but not Keras.

Is this expected behavior? If so, the warning should be suppressed for this case. If not, the documentation should be updated to remove mention of native support for Keras models.