HuangJunJie2017 / BEVDet

Code base of the BEVDet series .
Apache License 2.0
1.45k stars 266 forks source link

pkl文件 #367

Open 1yuechu1 opened 4 weeks ago

1yuechu1 commented 4 weeks ago

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. The bug has not been fixed in the latest version.

Describe the bug 当我按照示例代码生成pkl文件的时候,当运行到一定的iteration的时候,会出现opencv 报错,打印出来两者的格式发现都是numpy的形式

Environment

  1. Please run python mmdet3d/utils/collect_env.py to collect necessary environment information and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source]
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)
      
      sys.platform: linux
      Python: 3.8.20 (default, Oct  3 2024, 15:24:27) [GCC 11.2.0]
      CUDA available: True
      GPU 0: NVIDIA A800 80GB PCIe
      CUDA_HOME: /usr/local/cuda
      NVCC: Cuda compilation tools, release 11.6, V11.6.55
      GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
      PyTorch: 1.10.0+cu113
      PyTorch compiling details: PyTorch built with:
    • GCC 7.3
    • C++ Version: 201402
    • Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
    • Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
    • OpenMP 201511 (a.k.a. OpenMP 4.5)
    • LAPACK is enabled (usually provided by MKL)
    • NNPACK is enabled
    • CPU capability usage: AVX512
    • CUDA Runtime 11.3
    • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
    • CuDNN 8.2
    • Magma 2.5.2
    • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.11.0+cu113 OpenCV: 4.10.0 MMCV: 1.5.3 MMCV Compiler: GCC 7.3 MMCV CUDA Compiler: 11.3 MMDetection: 2.25.1 MMSegmentation: 0.25.0 MMDetection3D: 1.0.0rc4+26144be spconv2.0: True

**Error traceback**
If applicable, paste the error trackback here.

2024-10-22 15:36:34,297 - mmdet - INFO - Epoch [1][2750/3517] lr: 2.000e-04, eta: 14:59:39, time: 0.645, data_time: 0.020, memory: 7922, task0.loss_xy: 0.1238, task0.loss_z: 0.1062, task0.loss_whl: 0.2299, task0.loss_yaw: 0.3007, task0.loss_vel: 0.0611, task0.loss_heatmap: 2.2082, loss: 3.0299, grad_norm: 3.6961 Traceback (most recent call last): File "tools/train.py", line 277, in main() File "tools/train.py", line 266, in main train_model( File "/mnt/home/Code/BEV/BEVDet/mmdet3d/apis/train.py", line 344, in train_model train_detector( File "/mnt/home/Code/BEV/BEVDet/mmdet3d/apis/train.py", line 319, in train_detector runner.run(data_loaders, cfg.workflow) File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 130, in run epoch_runner(data_loaders[i], **kwargs) File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 47, in train for i, data_batch in enumerate(self.data_loader): File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data return self._process_data(data) File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data data.reraise() File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/torch/_utils.py", line 434, in reraise raise exception cv2.error: Caught error in DataLoader worker process 0. Original Traceback (most recent call last): File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/mnt/home/Code/BEV/BEVDet/mmdet3d/datasets/custom_3d.py", line 435, in getitem data = self.prepare_train_data(idx) File "/mnt/home/Code/BEV/BEVDet/mmdet3d/datasets/custom_3d.py", line 229, in prepare_train_data example = self.pipeline(input_dict) File "/mnt/home/Code/BEV/BEVDet/mmdet3d/datasets/pipelines/compose.py", line 49, in call data = t(data) File "/mnt/home/Code/BEV/BEVDet/mmdet3d/datasets/pipelines/loading.py", line 1157, in call results['img_inputs'] = self.get_inputs(results) File "/mnt/home/Code/BEV/BEVDet/mmdet3d/datasets/pipelines/loading.py", line 1105, in get_inputs imgs.append(self.normalize_img(img)) File "/mnt/home/Code/BEV/BEVDet/mmdet3d/datasets/pipelines/loading.py", line 853, in mmlabNormalize img = imnormalize(np.array(img), mean, std, torgb) File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/mmcv/image/photometric.py", line 22, in imnormalize return imnormalize(img, mean, std, torgb) File "/mnt/home/anaconda3/envs/bevdet/lib/python3.8/site-packages/mmcv/image/photometric.py", line 43, in imnormalize cv2.subtract(img, mean, img) # inplace cv2.error: OpenCV(4.10.0) /io/opencv/modules/core/src/arithm.cpp:658: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function 'arithm_op