Test detectors on an OOD task

deel-ai / oodeel

Simple, compact, and hackable post-hoc deep OOD detection for already trained tensorflow or pytorch image classifiers.

MIT License

52 stars 2 forks source link

To sum up

Previously, tests for baselines were very basic and only consisted in shapes tests.
Now, each baseline is evaluated on a 2D toy test case (4 blobs: 2 ID and 2 OOD => see figure below), where some assertions ensure that we get systematically an AUROC > 0.95 and an FPR95TPR < 0.05.
- Model chosen for tf and torch is a simple MLP (1 hidden layer) loaded from an url for reproducibility.
- Most important files: torch_methods_utils.py and tf_methods_utils.py (see eval_detector_on_blobs function).

blob_toy_dataset

☂️ Python Coverage

current status: ✅

Lines	Covered	Coverage	Threshold	Status
1614	1446	90%	70%	🟢

No new covered files...

File	Coverage	Status
oodeel/utils/tf_training_tools.py	71%	🟢
oodeel/utils/torch_training_tools.py	92%	🟢
TOTAL	82%	🟢

updated for commit: ea241f7 by action🐍