run run_relgan.py error

shenmadouyaowen commented 3 years ago

Describe the bug

python3 run_relgan.py 1 1 No modifications were made to the base config.

Environment

OS: [e.g., Ubuntu 18.04]
PyTorch = 1.7.0
Python 3.7
Numpy 1.14.5
CUDA 10.1
nltk 3.4

tqdm 4.32.1

Error traceback、

/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py:174: UserWarning: Output 0 of SplitWithSizesBackward is a view and is being modified inplace. This view is an output of a function that returns multiple views. Inplace operators on such views are being deprecated and will be forbidden starting from version 1.8. Consider using `unsafe_` version of the function that produced this view or don't modify this view inplace. (Triggered internally at  /pytorch/torch/csrc/autograd/variable.cpp:491.)
q *= (self.key_size ** -0.5)
[W python_anomaly_mode.cpp:104] Warning: Error detected in SplitWithSizesBackward. Traceback of forward call that caused the error:
File "main.py", line 169, in <module>
inst._run()
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 42, in _run
self.pretrain_generator(cfg.MLE_train_epoch)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 86, in pretrain_generator
pre_loss = self.train_gen_epoch(self.gen, self.train_data.loader, self.mle_criterion, self.gen_opt)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/instructor.py", line 103, in train_gen_epoch
pred = model.forward(inp, hidden)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/generator.py", line 51, in forward
out, hidden = self.lstm(emb, hidden)  # out: batch_size * seq_len * hidden_dim
File "/home/goscien/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 339, in forward
logit, memory = self.forward_step(inputs[:, idx_step], memory)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 307, in forward_step
next_memory = self.attend_over_memory(memory_plus_input)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 265, in attend_over_memory
attended_memory = self.multihead_attention(memory)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 171, in multihead_attention
q, k, v = torch.split(qkv_transpose, [self.key_size, self.key_size, self.value_size], -1)
File "/home/goscien/.local/lib/python3.7/site-packages/torch/functional.py", line 121, in split
return tensor.split(split_size_or_sections, dim)
File "/home/goscien/.local/lib/python3.7/site-packages/torch/tensor.py", line 476, in split
return super(Tensor, self).split_with_sizes(split_size, dim)
(function _print_stack)
Traceback (most recent call last):
File "main.py", line 169, in <module>
inst._run()
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 42, in _run
self.pretrain_generator(cfg.MLE_train_epoch)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 86, in pretrain_generator
pre_loss = self.train_gen_epoch(self.gen, self.train_data.loader, self.mle_criterion, self.gen_opt)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/instructor.py", line 105, in train_gen_epoch
self.optimize(optimizer, loss, model)
File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 143, in optimize
loss.backward(retain_graph=retain_graph)
File "/home/goscien/.local/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/goscien/.local/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [64, 2, 2, 768]], which is output 0 of PermuteBackward, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

I run python3 run_seqga.py 1 1normal I searched for a long time, but I didn't get the answer I wanted. What file do I need to modify to solve this error? thank you

AlexiaJM commented 3 years ago

@shenmadouyaowen

https://github.com/williamSYSU/TextGAN-PyTorch/blob/891635af6845edfee382de147faa4fc00c7e90eb/models/relational_rnn_general.py#L174

Solution is to change the line to: q = q * (self.key_size ** -0.5)

shenmadouyaowen commented 3 years ago

@shenmadouyaowen

https://github.com/williamSYSU/TextGAN-PyTorch/blob/891635af6845edfee382de147faa4fc00c7e90eb/models/relational_rnn_general.py#L174

解决方案是将行更改为：q = q *（self.key_size ** -0.5）

Thank you very much for your reply. It will be solved according to your revised method

williamSYSU / TextGAN-PyTorch