search

pmixer / SASRec.pytorch

PyTorch(1.6+) implementation of https://github.com/kang205/SASRec
Apache License 2.0
331 stars 90 forks source link

Best way to interpret final `matmul`? #26

Open jeffreymei opened 2 years ago

jeffreymei commented 2 years ago

In this line, what is the best way to think about this matmul? I see that it is calculating dot products for the final_feat and each emb in item_embs. If item_embs were normalized, then I could see this being essentially evaluating the cosine similarity (within a scaling factor) of the item_embs with respect to final_feat, but because the item_embs can vary in magnitude by ~30% or so it is not quite the same. Can you give any insight into this?

Thanks!

pmixer commented 2 years ago

Hi @jeffreymei , thx for the question, as I just reimplemented the SASRec in pytorch, some details may need to be double checked by original paper authors https://github.com/kang205/SASRec/issues.

From my view, the final matmul is kind of like evaluating the cosine similarity, as inherited from classical matrix factorization method for recommendation https://developers.google.com/machine-learning/recommendation/collaborative/matrix, w/o theoretical guarantee, people just want to have something to compare item distance, so just picked an older component to try.

It may not be the best choice, and elaborating into details may help finding something better for replacement.

Thus, instead of trying to interpret the final matmul accurately, personally, I suggest try something new beyong current approach.

Regards, Zan