agemagician / ProtTrans

ProtTrans is providing state of the art pretrained language models for proteins. ProtTrans was trained on thousands of GPUs from Summit and hundreds of Google TPUs using Transformers Models.
Academic Free License v3.0
1.13k stars 153 forks source link

RuntimeError: "addmm_impl_cpu_" not implemented for 'Half' #130

Closed Mjvolk3 closed 1 year ago

Mjvolk3 commented 1 year ago

If I change the colab runtime to in the colab notebook to cpu I get the following error.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[<ipython-input-7-23c8dfb81f21>](https://localhost:8080/#) in <cell line: 11>()
     10 # generate embeddings
     11 with torch.no_grad():
---> 12     embedding_repr = model(input_ids=input_ids,attention_mask=attention_mask)
     13 
     14 # extract embeddings for the first ([0,:]) sequence in the batch while removing padded & special tokens ([0,:7])

11 frames
[/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py](https://localhost:8080/#) in forward(self, input)
    112 
    113     def forward(self, input: Tensor) -> Tensor:
--> 114         return F.linear(input, self.weight, self.bias)
    115 
    116     def extra_repr(self) -> str:

RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

I thought this issue would be taken care of by the following line.

model.full() if device=='cpu' else model.half() # only cast to full-precision if no GPU is available

Could you help me with using the model on a cpu? Thanks.

mheinzinger commented 1 year ago

Hi, I have to admit that I only ran very limited experiments on CPU but you are right, I get the same error now. And you are also correct in saying that the line you quoted should handle exactly this: casting the model to full precision to make it work on CPU as well. I only now realized that the if does not evaluate to what I expected it to evaluate to (it never enters the model.full() branch). As a quick fix, you can simply cast the model to full precision via: model.to(torch.float32) (Maybe this functionality changed with more recent versions of huggingface/transformers but the model.full() function does not work anymore).

mheinzinger commented 1 year ago

I've now also updated the colab accordingly. Should work out-of-the-box now