search

THUNLP-MT / THUMT

An open-source neural machine translation toolkit developed by Tsinghua Natural Language Processing Group
BSD 3-Clause "New" or "Revised" License
701 stars 197 forks source link

Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! #106

Open treeson-li opened 3 years ago

treeson-li commented 3 years ago

I use Py-THUMT as the framework of new NMT algorithm. I only replace Transformer with another module and other parts remain unchanged. I use THUMT add_name() to add parameters to variables pool. When I run the code, I encount the following error. I don't know what reason cause the error. Would you give me some advice?

Traceback (most recent call last): File "../neuron/thumt/bin/trainer.py", line 528, in cli_main() File "../neuron/thumt/bin/trainer.py", line 524, in cli_main process_fn(0, parsed_args) File "../neuron/thumt/bin/trainer.py", line 502, in process_fn main(local_args) File "../neuron/thumt/bin/trainer.py", line 467, in main optimizer.apply_gradients(grads_and_vars) File "/data/code/nmt/neuron/thumt/optimizers/optimizers.py", line 480, in apply_gradients self._optimizer.apply_gradients(zip(grads, var_list)) File "/data/code/nmt/neuron/thumt/optimizers/optimizers.py", line 231, in applygradients m.mul(beta1).add(grad, alpha=1 - beta_1) RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

treeson-li commented 2 years ago

This bug is caused by tensors defined by torch.FloatTensor(), while all tensors in the original framework are defined by torch.Tensor(). Why this happens is unknown.