search

open-mmlab / mmaction2

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark
https://mmaction2.readthedocs.io
Apache License 2.0
4.15k stars 1.22k forks source link

Error on demo_spatiotemporal_det.py #1653

Open dnvnxg opened 2 years ago

dnvnxg commented 2 years ago

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. The bug has not been fixed in the latest version.

Describe the bug

I can't infer with LFB using the demo_spatiotemporal_det.py demo file.

Reproduction

  1. What command or script did you run?
python demo/demo_spatiotemporal_det.py --video testvid.mp4 --config configs/detection/lfb/lfb_nl_kinetics_pretrained_slowonly_r50_4x16x1_20e_ava_rgb.py --checkpoint work_dirs/lfb/lfb_nl_kinetics_pretrained_slowonly_r50_4x16x1_20e_ava_rgb/best_mAP@0.5IOU_epoch_20.pth --det-score-thr 0.9 --action-score-thr 0.5 --label-map tools/data/ava/label_map.txt --predict-stepsize 8 --output-stepsize 4 --output-fps
  1. Did you make any modifications on the code or config? Did you understand what you have modified? I did not make any modifications to the configs.
  2. What dataset did you use? AVA

Environment

  1. Please run PYTHONPATH=${PWD}:$PYTHONPATH python mmaction/utils/collect_env.py to collect necessary environment information and paste it here. 
    
sys.platform: linux
Python: 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0]
CUDA available: True
GPU 0,1,2,3,4,5,6,7: NVIDIA RTX A6000
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.5, V11.5.119
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
PyTorch: 1.11.0
PyTorch compiling details: PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.5.2 (Git Hash a9302535553c73243c632ad3c4c80beec3d19a1e)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.3
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
- CuDNN 8.2
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.12.0 OpenCV: 4.5.5 MMCV: 1.5.0 MMCV Compiler: GCC 9.4 MMCV CUDA Compiler: 11.5 MMAction2: 0.24.0+c7b45f0

3. You may add addition that may be helpful for locating the problem, such as
   - How you installed PyTorch \[e.g., pip, conda, source\]
   - Other environment variables that may be related (such as `$PATH`, `$LD_LIBRARY_PATH`, `$PYTHONPATH`, etc.)

   I installed pytorch via conda

**Error traceback**

If applicable, paste the error traceback here.

load checkpoint from http path: http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth Performing Human Detection for each frame [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 185/185, 14.1 task/s, elapsed: 13s, ETA: 0sLoading LFB from data/ava/lfb_half/lfb_train.pkl... Loading LFB from data/ava/lfb_half/lfb_val.pkl... LFB has been loaded on cuda:0. load checkpoint from local path: work_dirs/lfb/lfb_nl_kinetics_pretrained_slowonly_r50_4x16x1_20e_ava_rgb/best_mAP@0.5IOU_epoch_20.pth Performing SpatioTemporal Action Detection for each clip [ ] 0/185, elapsed: 0s, ETA:Traceback (most recent call last): File "/ascldap/users/dxgrieg/mmaction2/demo/demo_spatiotemporal_det.py", line 421, in main() File "/ascldap/users/dxgrieg/mmaction2/demo/demo_spatiotemporal_det.py", line 372, in main result = model( File "/ascldap/users/dxgrieg/anaconda3/envs/open-mmlab/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, kwargs) File "/ascldap/users/dxgrieg/anaconda3/envs/open-mmlab/lib/python3.10/site-packages/mmcv/runner/fp16_utils.py", line 110, in new_func return old_func(args, kwargs) File "/ascldap/users/dxgrieg/mmaction2/mmdetection/mmdet/models/detectors/base.py", line 174, in forward return self.forward_test(img, img_metas, kwargs) File "/ascldap/users/dxgrieg/mmaction2/mmdetection/mmdet/models/detectors/fast_rcnn.py", line 51, in forward_test return self.simple_test(imgs[0], img_metas[0], proposals[0], File "/ascldap/users/dxgrieg/mmaction2/mmdetection/mmdet/models/detectors/two_stage.py", line 183, in simple_test return self.roi_head.simple_test( File "/ascldap/users/dxgrieg/mmaction2/mmaction/models/heads/roi_head.py", line 79, in simple_test det_bboxes, det_labels = self.simple_test_bboxes( File "/ascldap/users/dxgrieg/mmaction2/mmaction/models/heads/roi_head.py", line 96, in simple_test_bboxes bbox_results = self._bbox_forward(x, rois, img_metas) File "/ascldap/users/dxgrieg/mmaction2/mmaction/models/heads/roi_head.py", line 33, in _bbox_forward bbox_feat = self.shared_head( File "/ascldap/users/dxgrieg/anaconda3/envs/open-mmlab/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(input, kwargs) File "/ascldap/users/dxgrieg/mmaction2/mmaction/models/heads/fbo_head.py", line 392, in forward lt_feat = self.sample_lfb(rois, img_metas).to(st_feat.device) File "/ascldap/users/dxgrieg/mmaction2/mmaction/models/heads/fbo_head.py", line 379, in sample_lfb lt_feat_list.append(self.lfb[img_metas[ind]['img_key']].to()) KeyError: 'img_key

kennymckormick commented 2 years ago

Hi, Xylobyte, You can not use LFB for demo unless you have completed its training and testing successfully, which means you need to generate the necessary files (i.e. the feature bank). If you think that's cumbersome, please use other configs for demo.

dnvnxg commented 2 years ago

Hey @kennymckormick I have completed training and inference of the feature bank and I run into this issue using my trained checkpoint

DreamyRainforest commented 1 year ago

Hello, I have the same problem with you. Have you solved it?