search

talreiss / Mean-Shifted-Anomaly-Detection

Mean-Shifted Contrastive Loss for Anomaly Detection (AAAI 2023)
https://arxiv.org/pdf/2106.03844.pdf
Other
117 stars 23 forks source link

Cannot reproduce results #1

Closed denix56 closed 3 years ago

denix56 commented 3 years ago

Hi,

I am trying to reproduce your results. However, the auc score stays nearly on the same level as in the beginning (0.7-0.8), and even decreases a bit. I tried it for label 0 and for label 2. The only difference from your default settings is batch size = 32

Also, I found that you are using Euclidean center loss instead of angular center loss. Could you please explain the reason of using it? I tried to replace the center loss with the angular one. However the results do not change noticably. The network does not converge.

SGD (lr=1e-5, weight_decay=5e-5)

talreiss commented 3 years ago

Hi,

I ran again CIFAR-10 on our end, and was able to reproduce the results. Please re-check if you changed the code, since at the beginning, with the initialized pre-trained model the results are 0.9+.

Regarding, the Euclidean vs. Angular losses. Please note that optimizing the Euclidean metric when dealing with unit vectors is proportional to optimizing the angular distance multiplied by a constant factor. Therefore, this optimization process is equivalent. Hope this clarifies your question.

denix56 commented 3 years ago

Thanks for your answer. Indeed, it was error in my code. While copying your center initialization I didn't set the model.eval() before