search

daintlab / unknown-detection-benchmarks

MIT License
12 stars 2 forks source link

EDL model's accuracy is so low #3

Open RuoyuChen10 opened 2 years ago

RuoyuChen10 commented 2 years ago

Thanks for your good job, in cifar-benchmark, I test several models is ok, however, when I test edl model, seems that the accuracy is not very high, very very low:

cifar-40 on augmix is 75.625 ok, on edl only 2.5??

I don't modify any parameters, except the --annealing I set as 10, I wanna how to set it to achieve the near accuracy of the paper?

Thanks, looking forward to your reply!

RuoyuChen10 commented 2 years ago

I may solve the problem, that's the function is different from the original paper.

In original paper, the annealing coefficient is min(1, epoch/annealing), however, here is directly set, with no decay.

You may set 0.1 as ok.

However, I also wanner what you set, 0.2 or any other?

Thanks.

JihyoKim00 commented 2 years ago

Yes you are right. The original paper sets the annealing coefficient like you said, but it cannot yield acceptable accuracy on my datasets which have more classes than the datasets used in the original paper. So I fixed it and searched appropriate values. I got 74.40% of accuracy with 0.003 on the CIFAR-10 benchmark. It is the same accuracy to the vanilla classifier trained with the same dataset. Other values can give similar accuracy.