Closed rgzn-aiyun closed 2 years ago
Can you please share a minimal example of code that causes this, and which versions of torch and transformers you are using?
Can you please share a minimal example of code that causes this, and which versions of torch and transformers you are using?
python example_usage.py PyTorch 1.11
Use the cpu to load, is that the case?
It seems mixed precision, which the .half() / float16 version of the model uses, isn't well-supported on CPU, so the float16 version of the 6B model can only run on GPU.
I've pushed a change that only loads float16 if CUDA == True.
Thanks for catching!
Weird?
return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) RuntimeError: expected scalar type Float but found Half