keras-uncertainty

[Examples| Tutorials| Docs]

Utilities and Models to perform Uncertainty Quantification on Keras.

Keras-Uncertainty is a high-level API to perform uncertainty quantification of machine learning models built with Keras. Many real-world applications not only require a model to make a prediction, but also to provide a confidence value that indicates how trustworthy is the prediction, which can be integrated into the decision making progress.

Typical research in machine learning applications (Computer Vision, NLP, etc) usually does not consider ways to produce well behaved uncertainty estimates, and machine learning methods can be used to extract or include uncertainty information into the model.

Classification Uncertainty

Code example available here

Regression Uncertainty

Code example available here

Installation

You can easily install with pip, using the following command:

pip install --user git+https://github.com/mvaldenegro/keras-uncertainty.git

Features

• Entropy and Negative Log-Likelihood metrics.
• Calibration plots for classification.
• Accuracy vs Confidence plot for classification.
• Statistical metrics like Interval Score and others.

Currently Implemented Methods

Please note that some UQ methods do not support some tasks like regression. Most methods support classification.