search

theeluwin / pytorch-sgns

Skipgram Negative Sampling implemented in PyTorch
MIT License
302 stars 59 forks source link

Bug in the Loss Function #12

Closed saketguru closed 4 years ago

saketguru commented 4 years ago

The loss function currently implement is -(oloss + nloss).mean()

It should be (-oloss + nloss).mean()

You want to minimize the distance between "positive samples" and maximize the distance between "negative samples".

theeluwin commented 4 years ago

As you would've noticed, there's .neg() chain at the end of nvectors definition (https://github.com/theeluwin/pytorch-sgns/blob/master/model.py#L72). But this does seem confusing. I'll fix it to be more verbose someday.

saketguru commented 4 years ago

Yes. I noticed it. Thanks for the response.