search

ModelOriented / DALEX

moDel Agnostic Language for Exploration and eXplanation
https://dalex.drwhy.ai
GNU General Public License v3.0
1.35k stars 166 forks source link

Multi class Feature Importance #338

Open PhilipNell opened 3 years ago

PhilipNell commented 3 years ago

Multi class feature importance is returning NaN for all variables. Does Dalex in Python support multi class? Jupyter NB and datafile is attached Archive.zip

hbaniecki commented 3 years ago

Hi, Python dalex doesn't support multiclass overall.

For it to work (in Feature Importance) you have to pass a custom predict_function to the Explainer and a custom loss_function to the model_parts method. Moreover, the form of y parameter should be taken into account in the loss_function.

multiclass.zip

Now, the predict_function doesn't return a (1d) np.array so other methods may fail. Also, dx.Explainer pointed out an error in residual_function (should probably be custom as well).

Multiclass in this package is not a priority as all (or most) of the methods would need to be customised EDIT: e.g. for predict_parts it would be plotting one plot per class and for model/predict_profile plotting one line per class. It can now be achieved by producing k times Explainer object, each predicting other class (custom predict_function), and plotting them together.

In the long run, it is something we definitely want to add.

PhilipNell commented 3 years ago

Awesome, thanks for the swift reply. Will have a look at it.

pbiecek commented 2 years ago

@hbaniecki would an additional argument like predict_function_target_column be a help here? then for multiclass object dalex can turn the classification into a binary classification model one-vs-all_others

hbaniecki commented 2 years ago

Sure, I just don't like the parameter name of predidct_function_target_column and am unsure about how to update all the yhat functions to utilize that.

hbaniecki commented 2 years ago

Example https://dalex.drwhy.ai/python-dalex-multioutput added in https://github.com/ModelOriented/DALEX-docs/commit/47378806fa9d32b612fb84adb46640638956ede7.