Open simaiden opened 4 years ago
Cosine Similarity is not implemented in the code. Instead, the weight computing process was replaced as applying Softmax on the inner product of z and m. I tried modified it to the way as written in the paper, however I found using cosine similarity only got me to an attention weight with all elements down to zero. So I guess the cosine distance measure was not applicable as it was said in the paper, which explained why they replaced it with inner product similarity in the actual implementation.
In the paper at section 3.3.2, in eq 4 shows that in MemAE each weight is computed using softmax operation and cosine similarity, but I can't find this in the code, so where this operation actually is used?
Thanks