search

zaiweizhang / H3DNet

MIT License
211 stars 25 forks source link

RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false #20

Closed giangdip2410 closed 3 years ago

giangdip2410 commented 3 years ago

EPOCH 000 Current learning rate: 0.010000 Current BN decay momentum: 0.500000 2021-03-17 08:52:17.518997 Traceback (most recent call last): File "train.py", line 379, in train(start_epoch) File "train.py", line 358, in train train_one_epoch() File "train.py", line 263, in train_one_epoch loss.backward() File "/usr/local/lib/python3.6/dist-packages/torch/tensor.py", line 195, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/usr/local/lib/python3.6/dist-packages/torch/autograd/init.py", line 99, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

I tried to reproduce as your instruction but the bug happens, do you have any idea why it did ? Thank you very much.

zaiweizhang commented 3 years ago

This is rarely hard to parse. It might be something wrong with the environment. Have you tried to train any other well-known models? For example, my codebase is built on this repo. Can you try it first? It's more easier to debug since it's simpler and lots of people have been using it.

giangdip2410 commented 3 years ago

@zaiweizhang : thank you very much for your suggestion, I will try with Votenet first.

giangdip2410 commented 3 years ago

@zaiweizhang : I uninstall pytorch==1.4.0 and install pytorch==1.2.0, then the bug disappear :) . Environment from PointNet++ to VoteNet, to H3DNet is very difficult to customize due to extend library of pointnet write by C++ , I think ?

zaiweizhang commented 3 years ago

It depends. Sampling and grouping are done in C++. For architecture changes, you can look into the pointnet2_modules.py file.

zaiweizhang commented 3 years ago

Feel free to reopen if you have follow up questions.