Does the Dalex offers a tradeoff between the interpretation and accuracy provided by a model?

[Nehagupta90]

Does the Dalex offers a tradeoff between the interpretation and accuracy provided by a model? Like the fairmodels package provide a function where we can get a tradeoff between the fairness and accuracy of a particular model.

Warm regards

[pbiecek]

There are no good measures to describe the interpretability of a model in a model-independent way. If you are looking for inspiration e.g. for tree models, see e.g. figure 4 from https://www.sciencedirect.com/science/article/pii/S016792362100066X

[Nehagupta90]

@Przemyslaw Biecek

By reading the abstract and a few other details of the paper you provided, I got an idea that the authors believe interpretability of the blackbox models using XAI is not very important (useful). Am I wrong?

How can we find if the interpretability (e.g. via the Dalex package) has impacted the performance of the models?

Thank you

On Wed, Feb 16, 2022 at 4:31 PM Przemysław Biecek @.***> wrote:

There are no good measures to describe the interpretability of a model in a model-independent way. If you are looking for inspiration e.g. for tree models, see e.g. figure 4 from https://www.sciencedirect.com/science/article/pii/S016792362100066X

— Reply to this email directly, view it on GitHub https://github.com/ModelOriented/DALEX/issues/479#issuecomment-1041699531, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2ZZ2I363OLD3FX6QRFAITU3O7OZANCNFSM5OPBQUAQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

[pbiecek]

Interpretability is important. But I disagree with the popular belief that there is a tradeoff between interpretability and performance. For most people, a linear model with 10 variables is completely unpredictable, even though it would seem that linear models have a simple structure. So the division between interpretable and uninterpretable models is artificial today.

I am much closer to the approach of using XAI techniques to understand the relevant behaviour of models in order to create better and better models. So XAI is a tool for knowledge extraction and model validation. I don't have to sacrifice anything to use techniques like SHAP or Break-Down, they should be applied to every models.

[Nehagupta90]

@Przemyslaw Biecek

Do you believe evaluating the performance of the models (and then selecting the best) is mandatory before we provide explanations to the models? In our recent paper (where we used an instance-level explanation), the reviewers raised objections like "what is the guarantee that the employed models provide better accuracy. Explanation of a least accurate model is misleading".

Do we need to first evaluate the models for accuracy before using the xai tools for the interpretation? I believe there should have a (Dalex) package which provides this accuracy-interpretation tradeoff. The fairmodels package does provide such a tool.

On Wed, Feb 16, 2022 at 10:48 PM Przemysław Biecek @.***> wrote:

Interpretability is important. But I disagree with the popular belief that there is a tradeoff between interpretability and performance. For most people, a linear model with 10 variables is completely unpredictable, even though it would seem that linear models have a simple structure. So the division between interpretable and uninterpretable models is artificial today.

I am much closer to the approach of using XAI techniques to understand the relevant behaviour of models in order to create better and better models. So XAI is a tool for knowledge extraction and model validation. I don't have to sacrifice anything to use techniques like SHAP or Break-Down, they should be applied to every models.

— Reply to this email directly, view it on GitHub https://github.com/ModelOriented/DALEX/issues/479#issuecomment-1042341679, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2ZZ2OEB7KEYWYWLGE56RDU3QLRLANCNFSM5OPBQUAQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

[hbaniecki]

Hi! Let's use GitHub issues to discuss matters related to the DALEX package and related tools.

Does the Dalex offers a tradeoff between the interpretation and accuracy provided by a model?

No. First, you would need to measure the interpretability of a model, which is not available in DALEX (mostly due to working with black-boxes in a model-agnostic fashion). For example, you could count the number of parameters in logistic regression/layers and neurons in a neural network, or measure the depth of decision trees/tree-ensembles.

Like the fairmodels package provide a function where we can get a tradeoff between the fairness and accuracy of a particular model.

Yes, because the fairmodels package aims to measure fairness. DALEX doesn't measure explain/interpretability.

Do we need to first evaluate the models for accuracy before using the xai tools for the interpretation?

This question seems too broad and depends on the context.

I believe there should have a (Dalex) package, which provides this accuracy-interpretation tradeoff. The fairmodels package does provide such a tool.

This is a cool idea, one would need to provide concrete interpretability measures first.

I hope my answer helped in the discussion :) Closing due to drifting offtopic.