Input expected to have 1 channel but got 6 channels instead

[BeMuCa]

While trying to run evaluation with:

torchpack dist-run -np 1 -v python tools/test.py configs/nuscenes/det/transfusion/secfpn/camera+lidar/swint_v0p075/convfuser.yaml pretrained/bevfusion-det.pth --out ./BEVFusion_results.pkl --eval mAP

I run into the error seen bellow, does anyone have a similiar problem?

I suspect it has something to do with the Radar changes, increasing the conv layer input size by 5, but not sure, yet.

• i have the exact prerequesites
• did create_data.py (which created "nuscenes_infos_val_radar.pkl - which I renamed and deleted the radar out)
• got newest bevfusion-det.pth file

Output:

return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] 2023-12-06 22:39:32,609 - mmdet - INFO - load checkpoint from http path: https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth load checkpoint from local path: pretrained/bevfusion-det.pth [ ] 0/6019, elapsed: 0s, ETA:Traceback (most recent call last):

File "tools/test.py", line 232, in main()

File "tools/test.py", line 205, in main outputs = multi_gpu_test(model, data_loader, args.tmpdir, args.gpu_collect)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/mmdet/apis/test.py", line 98, in multi_gpu_test result = model(return_loss=False, rescale=True, **data)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 886, in forward output = self.module(*inputs[0], **kwargs[0])

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 128, in new_func output = old_func(*new_args, **new_kwargs)

File "/home/berk/PycharmProjects/math-879-23-berk-hybrid-fusion/DeepFusion/BEVFusion_MIT_version/mmdet3d/models/fusion_models/bevfusion.py", line 253, in forward outputs = self.forward_single(

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 128, in new_func output = old_func(*new_args, **new_kwargs)

File "/home/berk/PycharmProjects/math-879-23-berk-hybrid-fusion/DeepFusion/BEVFusion_MIT_version/mmdet3d/models/fusion_models/bevfusion.py", line 301, in forward_single feature = self.extract_camera_features(

File "/home/berk/PycharmProjects/math-879-23-berk-hybrid-fusion/DeepFusion/BEVFusion_MIT_version/mmdet3d/models/fusion_models/bevfusion.py", line 133, in extract_camera_features x = self.encoders["camera"]["vtransform"](

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs)

File "/home/berk/PycharmProjects/math-879-23-berk-hybrid-fusion/DeepFusion/BEVFusion_MIT_version/mmdet3d/models/vtransforms/depth_lss.py", line 100, in forward x = super().forward(*args, **kwargs)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 214, in new_func output = old_func(*new_args, **new_kwargs)

File "/home/berk/PycharmProjects/math-879-23-berk-hybrid-fusion/DeepFusion/BEVFusion_MIT_version/mmdet3d/models/vtransforms/base.py", line 349, in forward x = self.get_cam_feats(img, depth, mats_dict)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 214, in new_func output = old_func(*new_args, **new_kwargs)

File "/home/berk/PycharmProjects/math-879-23-berk-hybrid-fusion/DeepFusion/BEVFusion_MIT_version/mmdet3d/models/vtransforms/depth_lss.py", line 88, in get_cam_feats d = self.dtransform(d)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/container.py", line 141, in forward input = module(input)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 446, in forward return self._conv_forward(input, self.weight, self.bias)

File "/home/berk/.conda/envs/BEVFusion/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward return F.conv2d(input, weight, bias, self.stride,

RuntimeError: Given groups=1, weight of size [8, 1, 1, 1], expected input[6, 6, 256, 704] to have 1 channels, but got 6 channels instead

Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted.

---

mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was:

Process name: [[47040,1],0] Exit code: 1

[BeMuCa]

https://github.com/mit-han-lab/bevfusion/issues/478 (https://github.com/mit-han-lab/bevfusion/issues/455)

Solved the problem for me!