embedding on GPU

[dami23]

The demo.ipynb and extract_feature.py in dir encoder/ are run on cpu. Could you give me a cue to run the two codes on GPU? Thank you for your help!

[aconneau]

Hi, could you try model = model.cuda()

in cell 2? Thanks, Alexis

[dami23]

@aconneau thank you for your reply, I tried model = model.cuda(), the error: Expected object of type torch.cuda.FloatTensor but found type torch.FloatTensor for argument #4 'mat1' from line "embeddings = model.encode(sentences, bsize=128, tokenize=False, verbose=True)". the problem is the sentences also need to be transformed to a tensor, but fromnumpy() can't convert type numpy.str to a tensor. So, could you give me a solution about this problem? Sincerely thank you for your kindness help!

[aconneau]

I tried on my side and it worked so I wasn't able to reproduce the issue. Could you output the full error message?

Which pytorch version are you using?

[dami23]

File "/informatik2/tams/home/mi/lieber_mi/projects/object_affordance/code/InferSent/demo.py", line 36, in embeddings = model.encode(sentences, bsize=128, tokenize=False, verbose=True) File "/informatik2/tams/home/mi/lieber_mi/projects/object_affordance/code/InferSent/models.py", line 226, in encode (batch, lengths[stidx:stidx + bsize])).data.numpy() File "/informatik2/tams/home/mi/lieber_mi/projects/object_affordance/code/InferSent/models.py", line 68, in forward sent_output = self.enc_lstm(sent_packed)[0] # seqlen x batch x 2nhid File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 491, in call result = self.forward(input, *kwargs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/rnn.py", line 192, in forward output, hidden = func(input, self.all_weights, hx, batch_sizes) File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 323, in forward return func(input, fargs, *fkwargs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 243, in forward nexth, output = func(input, hidden, weight, batch_sizes) File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 86, in forward hy, output = inner(input, hidden[l], weight[l], batch_sizes) File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 158, in forward hidden = inner(step_input, hidden, weight) File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 33, in LSTMCell gates = F.linear(input, w_ih, b_ih) + F.linear(hx, w_hh, b_hh) File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 992, in linear return torch.addmm(bias, input, weight.t()) RuntimeError: Expected object of type torch.cuda.FloatTensor but found type torch.FloatTensor for argument #4 'mat1'.

This is the full error. I use python3.5 and pytorch 0.4.0.

[aconneau]

Hi, I think I understand where the error comes from. In https://github.com/facebookresearch/InferSent/blob/master/models.py#L51 I check whether the model has been put on cuda or not. This might not work anymore since the new version of pytorch. Could you modify this line and print print(type(self.enc_lstm.bias_hh_l0.data)) right before line 51?

Thanks

[dami23]

the output is <class 'torch.Tensor'>. It is true, the model has not been put on cuda. I tried on pytorch 0.5.0, the error is same. Thank you for your help!

[dami23]

Maybe I find an "answer". I modified the line 51 in model.py to return 'cuda' (the latter part "in str(type(self.enc_lstm.bias_hh_l0.data))" is commented). And now, the embedding can run on GPU.

[aconneau]

So in the new pytorch version (>=0.4), the type of a Tensor which is on cuda is not "torch.cudaTensor" anymore but still "torch.Tensor", hence the error.

To fix the issue, could you modify the line : https://github.com/facebookresearch/InferSent/blob/master/models.py#L51 by return self.enc_lstm.bias_hh_l0.data.is_cuda and see if it works? Thanks

[dami23]

@aconneau your solution is the correct answer, it worked. Thank you for reply and help, best regards!!