search

open-mmlab / mmdetection3d

OpenMMLab's next-generation platform for general 3D object detection.
https://mmdetection3d.readthedocs.io/en/latest/
Apache License 2.0
5.18k stars 1.52k forks source link

[Bug] #2094

Open JiahaoXia opened 1 year ago

JiahaoXia commented 1 year ago

Prerequisite

Task

I have modified the scripts/configs, or I'm working on my own tasks/models/datasets.

Branch

master branch https://github.com/open-mmlab/mmdetection3d

Environment

sys.platform: linux
Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0]
CUDA available: True
GPU 0,1,2,3,4,5: Quadro RTX 5000
CUDA_HOME: /usr/local/cuda-11.6
NVCC: Cuda compilation tools, release 11.6, V11.6.124
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
PyTorch: 1.12.1
PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.3.2  (built against CUDA 11.5)
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

TorchVision: 0.13.1
OpenCV: 4.6.0
MMCV: 1.6.1
MMCV Compiler: GCC 9.3
MMCV CUDA Compiler: 11.3
MMDetection: 2.25.3
MMSegmentation: 0.29.0
MMDetection3D: 1.0.0rc5+962fc83
spconv2.0: False

Reproduces the problem - code sample

demo/mono_det_demo.py

Reproduces the problem - command or script

python demo/mono_det_demo.py demo/data/TrafficVideo/camera2.jpg demo/data/TrafficVideo/camera2.json configs/imvoxelnet/imvoxelnet_4x8_kitti-3d-car.py checkpoints/imvoxelnet_4x8_kitti-3d-car_20210830_003014-3d0ffdf4.pth --score-thr 0.2 --out-dir demo/imvoxelnet/

Reproduces the problem - error message

Traceback (most recent call last):
  File "demo/mono_det_demo.py", line 46, in <module>
    main()
  File "demo/mono_det_demo.py", line 33, in main
    result, data = inference_mono_3d_detector(model, args.image, args.ann)
  File "/home/rise/sdd2T/XJH/mmdetection3d/mmdet3d/apis/inference.py", line 246, in inference_mono_3d_detector
    box_type_3d, box_mode_3d = get_box_type(cfg.data.test.box_type_3d)
  File "/home/rise/Downloads/home/rise/Documents/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/utils/config.py", line 50, in __getattr__
    raise ex
AttributeError: 'ConfigDict' object has no attribute 'box_type_3d'

Additional information

I have added

box_type_3d='Camera'

to configure file imvoxelnet_4x8_kitti-3d-car.py however, I got other errors

Traceback (most recent call last):
  File "demo/mono_det_demo.py", line 46, in <module>
    main()
  File "demo/mono_det_demo.py", line 33, in main
    result, data = inference_mono_3d_detector(model, args.image, args.ann)
  File "/home/rise/sdd2T/XJH/mmdetection3d/mmdet3d/apis/inference.py", line 285, in inference_mono_3d_detector
    result = model(return_loss=False, rescale=True, **data)
  File "/home/rise/Downloads/home/rise/Documents/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/rise/Downloads/home/rise/Documents/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 116, in new_func
    return old_func(*args, **kwargs)
  File "/home/rise/Downloads/home/rise/Documents/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 174, in forward
    return self.forward_test(img, img_metas, **kwargs)
  File "/home/rise/sdd2T/XJH/mmdetection3d/mmdet3d/models/detectors/imvoxelnet.py", line 139, in forward_test
    return self.simple_test(img, img_metas)
  File "/home/rise/sdd2T/XJH/mmdetection3d/mmdet3d/models/detectors/imvoxelnet.py", line 151, in simple_test
    x, valid_preds = self.extract_feat(img, img_metas)
  File "/home/rise/sdd2T/XJH/mmdetection3d/mmdet3d/models/detectors/imvoxelnet.py", line 85, in extract_feat
    get_proj_mat_by_coord_type(img_meta, self.coord_type))
  File "/home/rise/sdd2T/XJH/mmdetection3d/mmdet3d/core/bbox/structures/utils.py", line 311, in get_proj_mat_by_coord_type
    return img_meta[mapping[coord_type]]
KeyError: 'lidar2img'
ZCMax commented 1 year ago

Can you provide the detailed config demo/data/TrafficVideo/camera2.json? We don't officially support running demo on ImVoxelNet.

JiahaoXia commented 1 year ago

@ZCMax This is the camera2.json

{
    "images": [
        {
            "file_name": "camera2.jpg",
            "cam_intrinsic": [
                [
                    1598.590308370044,
                    0.0,
                    1296
                ],
                [
                    0.0,
                    1591.578947368421,
                    972
                ],
                [
                    0.0,
                    0.0,
                    1.0
                ]
            ]
        }
    ]
}
Tai-Wang commented 1 year ago

ImVoxelNet is different from other monocular methods like FCOS3D. It needs the transformation between LiDAR and camera systems, i.e., extrinsic transformation, instead of only requiring intrinsic parameters in FCOS3D. You'll need to define your LiDAR system and align ImVoxelNet to your case. You can probably define a pseudo-LiDAR system aligned with the XYZ directions of the mmdet3d LiDAR system while aligning its origin with the camera system.