search

donggong1 / memae-anomaly-detection

MemAE for anomaly detection. -- Gong, Dong, et al. "Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection". ICCV 2019.
https://donggong1.github.io/anomdec-memae.html
MIT License
465 stars 103 forks source link

About Attention for Memory Addressig #12

Open simaiden opened 4 years ago

simaiden commented 4 years ago

In the paper at section 3.3.2, in eq 4 shows that in MemAE each weight is computed using softmax operation and cosine similarity, but I can't find this in the code, so where this operation actually is used?

Thanks

Wolfybox commented 4 years ago

Cosine Similarity is not implemented in the code. Instead, the weight computing process was replaced as applying Softmax on the inner product of z and m. I tried modified it to the way as written in the paper, however I found using cosine similarity only got me to an attention weight with all elements down to zero. So I guess the cosine distance measure was not applicable as it was said in the paper, which explained why they replaced it with inner product similarity in the actual implementation.