search

clovaai / FocusSeq2Seq

[EMNLP 2019] Mixture Content Selection for Diverse Sequence Generation (Question Generation / Abstractive Summarization)
https://arxiv.org/abs/1909.01953
MIT License
113 stars 20 forks source link

CUDNN_STATUS_EXECUTION_FAILED #7

Open aravindchaluvadi opened 4 years ago

aravindchaluvadi commented 4 years ago

I amfacing this issue while running the train.py command

I am using CUDA=10.0 and GPU= GeForce GTX 1660 Ti

COMMAND : python train.py --task=QG --model=NQG --load_glove=True --feature_rich --data=squad --rnn=GRU --dec_hidden_size=512 --dropout=0.5 --batch_size=64 --eval_batch_size=64 --use_focus=True --n_mixture=3 --decoding=greedy

==== Weight Initialization ====

Total Parameters: 12295050

Loaded word embedding! Traceback (most recent call last): File "train.py", line 110, in model.to(device) File "/home/xxx/FocusSeq2Seq/focus/lib/python3.6/site-packages/torch/nn/modules/module.py", line 386, in to return self._apply(convert) File "/home/xxxx/FocusSeq2Seq/focus/lib/python3.6/site-packages/torch/nn/modules/module.py", line 193, in _apply module._apply(fn) File "/home/xxxx/FocusSeq2Seq/focus/lib/python3.6/site-packages/torch/nn/modules/module.py", line 193, in _apply module._apply(fn) File "/home/xxxx/FocusSeq2Seq/focus/lib/python3.6/site-packages/torch/nn/modules/module.py", line 193, in _apply module._apply(fn) [Previous line repeated 1 more time] File "/home/xxxx/FocusSeq2Seq/focus/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 127, in _apply self.flatten_parameters() File "/home/xxx/FocusSeq2Seq/focus/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 123, in flatten_parameters self.batch_first, bool(self.bidirectional)) RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

j-min commented 4 years ago

I have not seen that cuDNN error. This might be an cuDNN compatibility issue for GTX 1660 Ti discussed here. Could you run other PyTorch code in your computer? Could you run this code in different hardward environment?