search

talreiss / Mean-Shifted-Anomaly-Detection

Mean-Shifted Contrastive Loss for Anomaly Detection (AAAI 2023)
https://arxiv.org/pdf/2106.03844.pdf
Other
117 stars 23 forks source link

Optimizer ablation study #12

Closed Doch88 closed 1 year ago

Doch88 commented 1 year ago

Hi! Thank you for the very insightful and useful paper. I was testing it with a custom dataset and I found that using Adam without weight decay, the loss is way more stable than with using SGD with weight decay. But I'm not sure if can affect in any way the learning or if it conflicts with some theoretical background around the paper. And actually with SGD I was observing a much faster collapse than with Adam (but I guess it can also depend on the choice of the hyperparameters). I was wondering if you guys have made some sort of ablation study regarding the optimizers or if there is a reasoning behind the choice of SGD. Thank you in advance!

talreiss commented 1 year ago

Hi, We do not have an ablation study available for the optimizer. We encourage you to experiment with it.

Tal