[Extension] Monitoring Model Deterioration with xAI uncertainty estimation

[cmougan]

Hi,

I will like to propose adding our last paper along with several other resources

Mougan, C. and Nielsen, D. S. (2023). Monitoring model deterioration with explainable uncertainty estimation via non-parametric bootstrap. In AAAI 2023 Conference on Artificial Intelligence.

Paper: https://arxiv.org/abs/2201.11676 Video: https://www.youtube.com/watch?v=Jx_Snwkf_nw&ab_channel=CarlosMougan Python package: https://github.com/saattrupdan/doubt Blogs:

• Using Uncertainty to Monitor ML models: https://saattrupdan.github.io/2022-11-19-monitoring-with-uncertainty/
• Bootstrapping prediction intervals https://saattrupdan.github.io/2020-02-20-confidence/
• Quantile Regression https://saattrupdan.github.io/2020-03-09-quantile-regression/
• Doubt: https://saattrupdan.github.io/2021-04-04-doubt/ Authors: Us

If you think this can be a great contribution, let me know and I do the PR

[valeman]

Hi Carlos,

Does this have Conformal Prediction in it?

Regards Valeriy

On Tue, Jan 17, 2023 at 1:32 PM Carlos Mougan @.***> wrote:

Hi,

I will like to propose adding our last paper along with several other resources

Mougan, C. and Nielsen, D. S. (2023). Monitoring model deterioration with explainable uncertainty estimation via non-parametric bootstrap. In AAAI 2023 Conference on Artificial Intelligence.

Paper: https://arxiv.org/abs/2201.11676 Video: https://www.youtube.com/watch?v=Jx_Snwkf_nw&ab_channel=CarlosMougan Python package: https://github.com/saattrupdan/doubt Blogs:

• Using Uncertainty to Monitor ML models: https://saattrupdan.github.io/2022-11-19-monitoring-with-uncertainty/
• Bootstrapping prediction intervals https://saattrupdan.github.io/2020-02-20-confidence/
• Quantile Regression https://saattrupdan.github.io/2020-03-09-quantile-regression/
• Doubt: https://saattrupdan.github.io/2021-04-04-doubt/ Authors: Us

If you think this can be a great contribution, let me know and I do the PR

— Reply to this email directly, view it on GitHub https://github.com/valeman/awesome-conformal-prediction/issues/13, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGH5I3WLQANAZBGYBQHWJ2DWS2NP7ANCNFSM6AAAAAAT53YBAI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[cmougan]

Hi Valeriy,

I think so. The prediction interval is calculated by model bootstrapping. A comparison that we do in the paper is with MAPIE using coverage

[valeman]

Hi Carlos, it does not look as advancing Conformal Prediction respectfully I can't ad it for this reason even though it looks like a great article and work. ACP is focusing on CP and it is a hard decision not to include works tangent to CP but need to keep ACP focused. Thanks for bringing it to my attention happy to share it.

[cmougan]

Definitely, you should stick to having only CP-related work. It is very amazing work

Sorry, but I am just trying to learn. Why is it not CP? Perhaps I am missing something

One could say that the main difference bt Jacknife+ of MAPIE and our approach is that we do bootstrap no?