search

furkanyesiler / move

PyTorch code for training and evaluating MOVE, musically-motivated version embeddings
GNU Affero General Public License v3.0
49 stars 9 forks source link

Cos distance performs better? #3

Open dansoutner opened 4 years ago

dansoutner commented 4 years ago

Small improvement tip: cos distance seems to performs better, when I tried on my data. Worth trying to implement both along euclidian and make more tests.

furkanyesiler commented 4 years ago

Hello @dansoutner!

Thanks for the tip. I actually experimented with removing the batch norm at the end and using cosine distance, but it decreased the results so I didn't push it further. Do you keep the batch norm and apply cosine distance?

dansoutner commented 4 years ago

Used output embedding as they are now in your code, just computed cosdist insted of euclidian. I tred this on two (relatively) small datasets and helps a little, but noticeable (MR 5.43 vs 5.86).

furkanyesiler commented 4 years ago

Ah okay, I thought you trained it with cosine distance as well. I'll add the option to use it for both training and evaluation.

Thanks again for the tip!