search

hendrycks / outlier-exposure

Deep Anomaly Detection with Outlier Exposure (ICLR 2019)
Apache License 2.0
541 stars 107 forks source link

Are the parameters for "300K Random Images " vs. "80m TinyImaget" same. #22

Open JimZAI opened 1 year ago

JimZAI commented 1 year ago

Hi, thank you for your insightful work.

I wonder Are the parameters for "300K Random Images " vs. "80m TinyImaget" same? When i reproduce the results, i find that directly replacing "80m TinyImaget" with "300K Random Images " brings significant performance degradation.

Thanks a lot.

utkutpcgl commented 1 year ago

Thanks for the contribution. I am also curious about this.

JimZAI commented 1 year ago

Could you please answer our questions. Thanks.

andyzoujm commented 1 year ago

Hi, the 300K images were randomly selected after excluding images from the CIFAR, Places, and LSUN classes, as well as images with divisive metadata, from the 80 Million Tiny Images dataset. No further modifications were made. Thank you.

JimZAI commented 1 year ago

Thanks for the response.

I would like to ask whether the traning hyperparameters (e.g. learning rate, ...) used for "300K Random Images " and "80m TinyImaget" are the same when performing model training. Because if the hyperparameters are not changed and just "300K Random Images" is used instead of "80m TinyImaget", the performance of the model degrades very significantly.

andyzoujm commented 1 year ago

We didn't tune any hyperparameter on 300K. Can you share some of the results you've obtained?