search

qiqihaer / 3DSSD-pytorch-openPCDet

The successful implementation of 3DSSD in Pytorch
Apache License 2.0
65 stars 19 forks source link

error when I train the model #3

Open czy-0326 opened 3 years ago

czy-0326 commented 3 years ago

2021-02-23 22:06:27,598 INFO **Start training kitti_models/3DSSD_openPCDet(3DSSD)** epochs: 0it [00:00, ?it/s] 2021-02-23 22:06:28,232 INFO **End training kitti_models/3DSSD_openPCDet(3DSSD)** When I run the command python train.py ,it doesnot work ,the problem is in line 3.

czy-0326 commented 3 years ago

when I delete the ckpt folder,it can continue ,but I meet the new problem 2021-02-23 22:25:04,173 INFO **Start training kitti_models/3DSSD_openPCDet(3DSSD)** epochs: 0%| | 0/80 [00:01<?, ?it/s] Traceback (most recent call last): | 0/3712 [00:00<?, ?it/s] File "train.py", line 198, in main() File "train.py", line 170, in main merge_all_iters_to_one_epoch=args.merge_all_iters_to_one_epoch File "/media/czy/新加卷/3DSSD-pytorch-openPCDet-main/tools/train_utils/train_utils.py", line 93, in train_model dataloader_iter=dataloader_iter File "/media/czy/新加卷/3DSSD-pytorch-openPCDet-main/tools/train_utils/train_utils.py", line 40, in train_one_epoch loss.backward() File "/home/czy/anaconda3/envs/3dssd/lib/python3.7/site-packages/torch/tensor.py", line 195, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/czy/anaconda3/envs/3dssd/lib/python3.7/site-packages/torch/autograd/init.py", line 99, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

Mian0429 commented 3 years ago

when I delete the ckpt folder,it can continue ,but I meet the new problem 2021-02-23 22:25:04,173 INFO **Start training kitti_models/3DSSD_openPCDet(3DSSD)** epochs: 0%| | 0/80 [00:01<?, ?it/s] Traceback (most recent call last): | 0/3712 [00:00<?, ?it/s] File "train.py", line 198, in main() File "train.py", line 170, in main merge_all_iters_to_one_epoch=args.merge_all_iters_to_one_epoch File "/media/czy/新加卷/3DSSD-pytorch-openPCDet-main/tools/train_utils/train_utils.py", line 93, in train_model dataloader_iter=dataloader_iter File "/media/czy/新加卷/3DSSD-pytorch-openPCDet-main/tools/train_utils/train_utils.py", line 40, in train_one_epoch loss.backward() File "/home/czy/anaconda3/envs/3dssd/lib/python3.7/site-packages/torch/tensor.py", line 195, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/czy/anaconda3/envs/3dssd/lib/python3.7/site-packages/torch/autograd/init.py", line 99, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

I met the same error as you when I train on cu101+pytorch1.4+python3.6, and it works when I use cu90+pytorch1.1+python3.6. I don't know why but it works.

GYE19970220 commented 2 years ago

same error，any answers?

3623687277 commented 10 months ago

same error，any answers?