search

nuclearboy95 / Anomaly-Detection-PatchSVDD-PyTorch

246 stars 56 forks source link

Question on SVDD' loss #2

Closed ArdalanM closed 4 years ago

ArdalanM commented 4 years ago

Hello,

Thank you for this insightful paper. I have a question regarding the SVDD' loss, equation (4) in your paper.

Lsvdd' ~ || f(p) - f(p') ||

What prevents your network for not returning zeros regardless p or p' (since there are no 'negative samples' in the loss) ? In your code, the loss tend to converge to zero eventually.

nuclearboy95 commented 4 years ago

Hi, thanks for your comment.

Lsvdd' converges to zero as long as SGD finds the "global minimum". Ruff et al. (Deep SVDD) prevented such trivial solution using some tricks (e.g., removing bias). However, we add self-supervised learning and the resulting loss becomes LpatchSVDD (equation 6).

ArdalanM commented 4 years ago

Thanks for the quick respond, it make sense now.

In order to prevent trivial solution (since bias term is learnt in your case) we should/must add the L_ssl term (equation 5).

Many thanks !