search

OATML / bdl-benchmarks

Bayesian Deep Learning Benchmarks
Apache License 2.0
663 stars 94 forks source link

[question] What is the random refer to in the diagrams #7

Closed kirk86 closed 4 years ago

kirk86 commented 4 years ago

I was wondering what does the random baseline refer to actually in tables/diagrams? Lastly, any idea why are ensembles performing worse than mc-drop, I think in the paper of deep ensembles they showed that they performed better than mc-drop (plz correct me if I'm wrong)

Thanks!

filangelos commented 4 years ago

The "random" baseline refers to (uniformly) random filtering of test data points, while the rest of the methods filter based on their uncertainty (i.e., filter the most k% uncertain points, where k the different x-axis thresholds).

Please refer to our paper for a discussion on the performance of the different methods.

kirk86 commented 4 years ago

Ευχαριστώ!