RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

[Mjvolk3]

If I change the colab runtime to in the colab notebook to cpu I get the following error.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[<ipython-input-7-23c8dfb81f21>](https://localhost:8080/#) in <cell line: 11>()
     10 # generate embeddings
     11 with torch.no_grad():
---> 12     embedding_repr = model(input_ids=input_ids,attention_mask=attention_mask)
     13 
     14 # extract embeddings for the first ([0,:]) sequence in the batch while removing padded & special tokens ([0,:7])

11 frames
[/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py](https://localhost:8080/#) in forward(self, input)
    112 
    113     def forward(self, input: Tensor) -> Tensor:
--> 114         return F.linear(input, self.weight, self.bias)
    115 
    116     def extra_repr(self) -> str:

RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

I thought this issue would be taken care of by the following line.

model.full() if device=='cpu' else model.half() # only cast to full-precision if no GPU is available

Could you help me with using the model on a cpu? Thanks.

[mheinzinger]

Hi, I have to admit that I only ran very limited experiments on CPU but you are right, I get the same error now. And you are also correct in saying that the line you quoted should handle exactly this: casting the model to full precision to make it work on CPU as well. I only now realized that the if does not evaluate to what I expected it to evaluate to (it never enters the model.full() branch). As a quick fix, you can simply cast the model to full precision via: model.to(torch.float32) (Maybe this functionality changed with more recent versions of huggingface/transformers but the model.full() function does not work anymore).

[mheinzinger]

I've now also updated the colab accordingly. Should work out-of-the-box now