search

utterworks / fast-bert

Super easy library for BERT based NLP models
Apache License 2.0
1.85k stars 342 forks source link

GeluCUDAKernelImpl" not implemented for 'Half' #177

Closed tbass134 closed 4 years ago

tbass134 commented 4 years ago

im working through the example project in the readme and running into this error:

RuntimeError: Caught RuntimeError in replica 0 on device 0. Original Traceback (most recent call last): File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker output = module(*input, kwargs) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, *kwargs) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 1176, in forward inputs_embeds=inputs_embeds, File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(input, kwargs) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 790, in forward encoder_attention_mask=encoder_extended_attention_mask, File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, kwargs) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 407, in forward hidden_states, attention_mask, head_mask[i], encoder_hidden_states, encoder_attention_mask File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, *kwargs) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 379, in forward intermediate_output = self.intermediate(attention_output) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(input, kwargs) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 332, in forward hidden_states = self.intermediate_act_fn(hidden_states) File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/functional.py", line 1126, in gelu return torch._C._nn.gelu(input) RuntimeError: "GeluCUDAKernelImpl" not implemented for 'Half'

I saw this issue in Apex https://github.com/NVIDIA/apex/issues/555 however, it looks like glu is already included..

yuchenlin commented 4 years ago

you can try to install pytorch nightly (1.5.0+) it works for me

tbass134 commented 4 years ago

Thanks @yuchenlin , that did the trick!