search

NolenChen / 3DStructurePoints

77 stars 15 forks source link

RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:383 #4

Closed QiangZiBro closed 4 years ago

QiangZiBro commented 4 years ago

My operations:

pip install -r requirements.txt
cd pointnet2
python setup.py build_ext --inplace
python train/train_structure_points.py -data_dir data/train -num_structure_points 16 -category plane -log_dir log

Error

then a RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:383 shows, I've seen this error for times, but I don't know what's the reason


train unsupervised structure points for bhcp

epoch: 0 ###########################
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=383 error=11 : invalid argument
Traceback (most recent call last):
  File "/home/qiangzibro/3DStructurePoints/train/train_structure_points.py", line 243, in <module>
    train(cmd_args=args)
  File "/home/qiangzibro/3DStructurePoints/train/train_structure_points.py", line 170, in train
    train_loss, train_cd_loss, train_consistent_loss, iters = train_one_epoch(model, optimizer, train_loader, lr_scheduler, bnm_scheduler, iters, criterions, num_of_trans=cmd_args.num_of_transform, num_inpts=cmd_args.num_inpts)
  File "/home/qiangzibro/3DStructurePoints/train/train_structure_points.py", line 61, in train_one_epoch
    structure_points_all = model(batch_points_all)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/qiangzibro/3DStructurePoints/models/pointnet2_structure_point_net.py", line 169, in forward
    xyz, features = module(xyz, features)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/qiangzibro/3DStructurePoints/pointnet2/utils/pointnet2_modules.py", line 64, in forward
    new_features = self.mlps[i](new_features)  # (B, mlp[-1], npoint, nsample)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/qiangzibro/anaconda3/envs/torch11/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 338, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:383
QiangZiBro commented 4 years ago

Update

I used conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch to install pytorch instead of pip install, error resolved