Closed giangdip2410 closed 3 years ago
This is rarely hard to parse. It might be something wrong with the environment. Have you tried to train any other well-known models? For example, my codebase is built on this repo. Can you try it first? It's more easier to debug since it's simpler and lots of people have been using it.
@zaiweizhang : thank you very much for your suggestion, I will try with Votenet first.
@zaiweizhang : I uninstall pytorch==1.4.0 and install pytorch==1.2.0, then the bug disappear :) . Environment from PointNet++ to VoteNet, to H3DNet is very difficult to customize due to extend library of pointnet write by C++ , I think ?
It depends. Sampling and grouping are done in C++. For architecture changes, you can look into the pointnet2_modules.py file.
Feel free to reopen if you have follow up questions.
EPOCH 000 Current learning rate: 0.010000 Current BN decay momentum: 0.500000 2021-03-17 08:52:17.518997 Traceback (most recent call last): File "train.py", line 379, in
train(start_epoch)
File "train.py", line 358, in train
train_one_epoch()
File "train.py", line 263, in train_one_epoch
loss.backward()
File "/usr/local/lib/python3.6/dist-packages/torch/tensor.py", line 195, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/usr/local/lib/python3.6/dist-packages/torch/autograd/init.py", line 99, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)
I tried to reproduce as your instruction but the bug happens, do you have any idea why it did ? Thank you very much.