Training Errors: "RuntimeError: Expected object of scalar type Float but got scalar type Half for argument #2 'mat2' in call to _th_mm_out"

[zlingBryan]

Hi, I ran into the following problems while training based on this project. Hope for help, thank you.

RUN: python extra_tools/train.py projects/configs/uvtr/lidar_based/uvtr_l_v0075_h5.py

Environment： TorchVision: 0.6.0a0+82fd1c8 OpenCV: 4.6.0 MMCV: 1.4.0 MMCV Compiler: GCC 5.4 MMCV CUDA Compiler: 10.1 MMDetection: 2.14.0 MMSegmentation: 0.14.1 MMDetection3D: 0.17.3+

Errors: 2022-08-12 11:21:31,523 - mmdet - INFO - Checkpoints will be saved to **/0812/mmdetection3d-0.17.3/work_dirs/uvtr_l_v0075_h5 by HardDiskBackend. Traceback (most recent call last): File "extra_tools/train.py", line 248, in main() File "extra_tools/train.py", line 244, in main meta=meta) File "**/0812/mmdetection3d-0.17.3/mmdet3d/apis/train.py", line 35, in train_model meta=meta) File "**/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector runner.run(data_loaders, cfg.workflow) File "**/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run epoch_runner(data_loaders[i], kwargs) File "****/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train self.run_iter(data_batch, train_mode=True, kwargs) File "****/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter kwargs) File "****/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 75, in train_step return self.module.train_step(*inputs[0], kwargs[0]) File "****/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 237, in train_step losses = self(data) File "****/.conda/envs/open-mmlab2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, kwargs) File "****/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 216, in new_func output = old_func(*new_args, new_kwargs) File "****/0812/mmdetection3d-0.17.3/projects/mmdet3d_plugin/models/detectors/uvtr.py", line 255, in forward return self.forward_train(kwargs) File "****/0812/mmdetection3d-0.17.3/projects/mmdet3d_plugin/models/detectors/uvtr.py", line 296, in forward_train pts_feat, img_feats, img_depth = self.extract_feat(points=points, img=img, img_metas=img_metas) File "**/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 130, in new_func output = old_func(*new_args, new_kwargs) File "****/0812/mmdetection3d-0.17.3/projects/mmdet3d_plugin/models/detectors/uvtr.py", line 187, in extract_feat pts_feats = self.extract_pts_feat(points, img_feats, img_metas) File "**/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 216, in new_func output = old_func(*new_args, new_kwargs) File "****/0812/mmdetection3d-0.17.3/projects/mmdet3d_plugin/models/detectors/uvtr.py", line 138, in extract_pts_feat x = self.pts_middle_encoder(voxel_features, coors, batch_size) File "**/.conda/envs/open-mmlab2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, kwargs) File "****/.conda/envs/open-mmlab2/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func return old_func(*args, kwargs) File "****/0812/mmdetection3d-0.17.3/projects/mmdet3d_plugin/models/pts_encoder/sparse_encoder_hd.py", line 119, in forward x = self.conv_input(input_sp_tensor) File "**/.conda/envs/open-mmlab2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, kwargs) File "****/0812/mmdetection3d-0.17.3/mmdet3d/ops/spconv/modules.py", line 130, in forward input = module(input) File "**/.conda/envs/open-mmlab2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, kwargs) File "****/0812/mmdetection3d-0.17.3/mmdet3d/ops/spconv/conv.py", line 186, in forward outids.shape[0]) File "**/0812/mmdetection3d-0.17.3/mmdet3d/ops/spconv/functional.py", line 65, in forward indice_pair_num, num_activate_out, False, True) File "**/0812/mmdetection3d-0.17.3/mmdet3d/ops/spconv/ops.py", line 124, in indice_conv int(subm)) RuntimeError: Expected object of scalar type Float but got scalar type Half for argument #2 'mat2' in call to _th_mm_out

[yanwei-li]

Hi, it seems an error of tensor type in Spconv. Do you use the Spconv 1.x or 2.x?

[zlingBryan]

There is no spconv in my environment. Should I install 1.x or 2.x?

[yanwei-li]

Hi, you need to install spconv 1.x if you want to use the pretrained models.