search

lonePatient / albert_pytorch

A Lite Bert For Self-Supervised Learning Language Representations
Apache License 2.0
710 stars 152 forks source link

CUDA error: no kernel image is available for execution on the device #27

Closed chiragsanghvi10 closed 4 years ago

chiragsanghvi10 commented 4 years ago

Hello,

I am getting the following error message

RuntimeError: CUDA error: no kernel image is available for execution on the device while executing sh run_classifier_stsb.sh I am working on STS-B dataset.

Input

STS-B dataset train.tsv

Output

Traceback (most recent call last): File "run_classifier.py", line 467, in <module> main() File "run_classifier.py", line 420, in main global_step, tr_loss = train(args, train_dataset, model, tokenizer) File "run_classifier.py", line 110, in train outputs = model(**inputs) File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__ result = self.forward(*input, **kwargs) File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward outputs = self.parallel_apply(replicas, inputs, kwargs) File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 85, in parallel_apply output.reraise() File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/_utils.py", line 385, in reraise raise self.exc_type(msg) RuntimeError: Caught RuntimeError in replica 1 on device 1. Original Traceback (most recent call last): File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker output = module(*input, **kwargs) File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__ result = self.forward(*input, **kwargs) File "/home/vv/git/albert_pytorch/albert_english_pytorch/model/modeling_albert.py", line 875, in forward head_mask=head_mask) File "/home/vv/git/albert_pytorch/albert_english_pytorch/alb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__ result = self.forward(*input, **kwargs) File "/home/vv/git/albert_pytorch/albert_english_pytorch/model/modeling_albert.py", line 594, in forward extended_attention_mask = extended_attention_mask.to(dtype=next(self.parameters()).dtype) # fp16 compatibility RuntimeError: CUDA error: no kernel image is available for execution on the device

Environment

CUDA Version: 10.1.105 PyTorch : 1.3.1 Python : 3.6.8 OS: Ubuntu 18.04.3 LTS x86_64 Kernel : 4.15.0-70-generic CPU : AMD Ryzen 7 2700X (16) @ 2.661GHz GPU : NVIDIA NVIDIA Corporation Device 1f08 GPU : NVIDIA GeForce GT 710

ANy updates on this? Thanks in advance.

lonePatient commented 4 years ago

@chiragsanghvi10， GT710? Maybe cuda and pytorch does not support GPU.