when i used the coco dataset to train a model,i missed this problem that about the cuda. while i used the CIHP dataset , it could work really well .I found some tips to solve this problem,for example ,add the code torch.cuda.set_device(1) to the loss.py while it seemed useless. So how to slove this problem ,thanks !
File "/data/parsing_env/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], kwargs[0])
File "/data/parsing_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, *kwargs)
File "/data/parsing/rcnn/modeling/model_builder.py", line 138, in forward
x, result, loss_uv = self.UV_RCNN(conv_features, result, targets)
File "/data/parsing_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(input, kwargs)
File "/data/parsing/rcnn/modeling/uv_rcnn/uv_rcnn.py", line 31, in forward
return self._forward_train(conv_features, proposals, targets)
File "/data/parsing/rcnn/modeling/uv_rcnn/uv_rcnn.py", line 43, in _forward_train
loss_Upoints, loss_Vpoints, loss_seg_AnnIndex, loss_IndexUVPoints = self.loss_evaluator(uv_logits)
File "/data/parsing/rcnn/modeling/uv_rcnn/loss.py", line 160, in call
torch.cuda.set_device(1)
File "/data/parsing_env/lib/python3.6/site-packages/torch/cuda/init.py", line 265, in set_device
torch._C._cuda_setDevice(device)
RuntimeError: cuda runtime error (10) : invalid device ordinal at /opt/conda/conda-bld/pytorch_1556653099582/work/torch/csrc/cuda/Module.cpp:33
when i used the coco dataset to train a model,i missed this problem that about the cuda. while i used the CIHP dataset , it could work really well .I found some tips to solve this problem,for example ,add the code torch.cuda.set_device(1) to the loss.py while it seemed useless. So how to slove this problem ,thanks !
File "/data/parsing_env/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward return self.module(*inputs[0], kwargs[0]) File "/data/parsing_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call result = self.forward(*input, *kwargs) File "/data/parsing/rcnn/modeling/model_builder.py", line 138, in forward x, result, loss_uv = self.UV_RCNN(conv_features, result, targets) File "/data/parsing_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call result = self.forward(input, kwargs) File "/data/parsing/rcnn/modeling/uv_rcnn/uv_rcnn.py", line 31, in forward return self._forward_train(conv_features, proposals, targets) File "/data/parsing/rcnn/modeling/uv_rcnn/uv_rcnn.py", line 43, in _forward_train loss_Upoints, loss_Vpoints, loss_seg_AnnIndex, loss_IndexUVPoints = self.loss_evaluator(uv_logits) File "/data/parsing/rcnn/modeling/uv_rcnn/loss.py", line 160, in call torch.cuda.set_device(1) File "/data/parsing_env/lib/python3.6/site-packages/torch/cuda/init.py", line 265, in set_device torch._C._cuda_setDevice(device) RuntimeError: cuda runtime error (10) : invalid device ordinal at /opt/conda/conda-bld/pytorch_1556653099582/work/torch/csrc/cuda/Module.cpp:33