search

nzw0301 / pytorch_skipgram

3 stars 2 forks source link

Make code faster #2

Closed nzw0301 closed 6 years ago

nzw0301 commented 6 years ago

I guess that the bottleneck is the negative sampling in NEG's forward function. So

  1. Use GPU sampler
  2. Before forwarding, I create neg samples and mini batch using multi prosess and data loader.

Both ways use more memory space than current implementation...

nzw0301 commented 6 years ago
  1. Pytorch does not support fast Categorical sampler such as AliasSampler in Chainer. So even if I create it, this code speed will be slow yet.
  2. explicit_main.py and /model/explicit_noise_sgns.py provide this version.