Unable to run inference using torch.distributed.launch

[Pixie8888]

Hi,

Thank you very much for sharing code! I follow the instruction to create environment: pytorch 1.8 + cu11.1 mmcv-full=1.2.7 mmdet=2.10.0 mmdet3d=0.8.0

But, when I run CUDA_VISIBLE_DEVICES=0 bash tools/dist_test.sh configs/imvoxelnet/imvoxelnet_scannet_fast.py checkpoints/imvoxelnet_scannet/20211007_113826.pth 1 --eval mAP, it gives the error:

ile "tools/test.py", line 153, in <module>
    main()
  File "tools/test.py", line 135, in main
    outputs = multi_gpu_test(model, data_loader, args.tmpdir,
  File "/home/yating/anaconda3/envs/imvoxelnet/lib/python3.8/site-packages/mmdet/apis/test.py", line 97, in multi_gpu_test
    result = model(return_loss=False, rescale=True, **data)
  File "/home/yating/anaconda3/envs/imvoxelnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yating/anaconda3/envs/imvoxelnet/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File "/home/yating/anaconda3/envs/imvoxelnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yating/anaconda3/envs/imvoxelnet/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
    return old_func(*args, **kwargs)
  File "/home/yating/anaconda3/envs/imvoxelnet/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 183, in forward
    return self.forward_test(img, img_metas, **kwargs)
  File "/home/yating/Documents/nerf/imvoxelnet/mmdet3d/models/detectors/imvoxelnet.py", line 91, in forward_test
    return self.simple_test(img, img_metas)
  File "/home/yating/Documents/nerf/imvoxelnet/mmdet3d/models/detectors/imvoxelnet.py", line 94, in simple_test
    x, valids, features_2d = self.extract_feat(img, img_metas, 'test')
  File "/home/yating/Documents/nerf/imvoxelnet/mmdet3d/models/detectors/imvoxelnet.py", line 46, in extract_feat
    batch_size = img.shape[0]
AttributeError: 'DataContainer' object has no attribute 'shape'

Note that this problem occurs regardless how many gpus I used. The training goes smoothly, but it will give the same error when doing validation. What should I do?

[filaPro]

May be try pytorch 1.6 with cuda 10? As I understand from provided logs we used these versions.

[Pixie8888]

Yeah I know, but some of packages I used only support python 3.10 (which corresponds to higher version of pytorch). Probably I have to re-implement everything without using any mm-packages.