On Out-of-Distribution Detection Algorithms with Deep Neural Skin CancerClassifiers
Code for the best paper of the ISIC Skin Image Analysis Workshop at CVPR 2020.
Paper Download: link
Workshop Talk: link
For the Mahalanobis, ODIN, and Baseline original codes, please, refer to the following repositories:
- ODIN: Out-of-Distribution Detector for Neural Networks
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
How to run
- Install the dependencies:
pip install -r requirements.txt - Download data and checkpoints running the
setup.shscript. - Run the network notebook (ex:
sk_mobilenet.ipynb) - The Mahalanobis, ODIN, and Baseline adaptations are available on mahalanobis_and_odin folder.
If you don't want to use the setup.shscript, you can download data from here and the checkpoints from here.
ISIC unknown detection is available on isic_submit folder.
Results
Unbiased performance in terms of TNR @ TPR 95%
Running time for Gram-OOD and Gram-OOD*
ISIC 2019 unknown detection - Resnet-50
Team on live leaderboard: Gram-OOD - UNK detection using Gram-OOD* - Resnet
The remaining results are available on this folder