search

hendrycks / ss-ood

Self-Supervised Learning for OOD Detection (NeurIPS 2019)
MIT License
264 stars 31 forks source link

Reproducing CIFAR-10 One-class OOD Detector with OE #14

Closed sehyungp92 closed 3 years ago

sehyungp92 commented 4 years ago

Firstly many thanks for sharing your work.

I found the paper very interesting and wanted to see if I could reproduce some of the results, especially regarding the one-class OOD detector on CIFAR-10 using OE. I was just wondering if this would be made available at any point.

Kind regards, Se

fourk-ty commented 4 years ago

@sehyungp92 did you reproduce the results of the multiclass_ood detector now? When I run the code the first time, the AUROC is only 0.54718556.

hendrycks commented 4 years ago

Did the network train properly? If it has reasonable classification accuracy its AUROC shouldn't be nearly that low.

On Mon, Jul 13, 2020 at 7:03 PM Fourk_tx notifications@github.com wrote:

@sehyungp92 https://github.com/sehyungp92 did you reproduce the results of the multiclass_ood detector now? When I run the code the first time, the AUROC is only 0.54718556.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hendrycks/ss-ood/issues/14#issuecomment-657922144, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZBITUGO6S4WY2PTREOTGTR3O4GZANCNFSM4OD4MJHA .

fourk-ty commented 4 years ago

Hi, @hendrycks , I thought it was not trained properly, but I did not change the hyper-parameters. Can you share your hyper-parameters to us?

Thanks for your reply.

hendrycks commented 4 years ago

It looks like the default hyperparamter is to train for 5 epochs. That should probably be 100 (sorry this code was added long after the experiments were run).

On Mon, Jul 13, 2020 at 7:21 PM Fourk_tx notifications@github.com wrote:

Hi, @hendrycks https://github.com/hendrycks , I thought it was not trained properly, but I did not change the hyper-parameters. Can you share your hyper-parameters to us?

Thanks for your reply.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hendrycks/ss-ood/issues/14#issuecomment-657926785, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZBITV46GNTMHMGMN4YG3DR3O6I7ANCNFSM4OD4MJHA .