920232796
commented
1 year ago First, you can switch the env from "DDP" to "pytorch", using single GPU or CPU to run the code. Then you also need to check the training data.
Open zzh1230 opened 1 year ago
920232796
commented
1 year ago First, you can switch the env from "DDP" to "pytorch", using single GPU or CPU to run the code. Then you also need to check the training data.
When I executed the train.py, I encountered the following error, how can I solve it?
Traceback (most recent call last): File "train.py", line 192, in
trainer.train(train_dataset=train_ds, val_dataset=val_ds)
File "/home/hdc/zjh/Diff-UNet-main/LiTS/light_training/trainer.py", line 262, in train
self.train_epoch(
File "/home/hdc/zjh/Diff-UNet-main/LiTS/light_training/trainer.py", line 361, in train_epoch
for idx, batch in enumerate(loader):
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in next
data = self._next_data()
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1182, in _next_data
idx, data = self._get_data()
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1148, in _get_data
success, data = self._try_get_data()
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 986, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/multiprocessing/queues.py", line 116, in get
return _ForkingPickler.loads(res)
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/multiprocessing/reductions.py", line 282, in rebuild_storage_fd
fd = df.detach()
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/multiprocessing/resource_sharer.py", line 58, in detach
return reduction.recv_handle(conn)
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/multiprocessing/reduction.py", line 189, in recv_handle
return recvfds(s, 1)[0]
File "/home/hdc/anaconda3/envs/pytorch/lib/python3.8/multiprocessing/reduction.py", line 164, in recvfds
raise RuntimeError('received %d items of ancdata' %