demo_skeleton.py error - Githubissues

open-mmlab / mmaction2

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

https://mmaction2.readthedocs.io

Apache License 2.0

4.29k stars 1.25k forks source link

demo_skeleton.py error #2366

Closed Ishihara-Masabumi closed 1 year ago

Ishihara-Masabumi commented 1 year ago

The doc issue

When I run demo_skeleton.py according to https://github.com/open-mmlab/mmaction2/tree/main/demo, the following error occurred.

$ python3 demo/demo_skeleton.py demo/demo_skeleton.mp4 demo/demo_skeleton_out.mp4 --config configs/skeleton/posec3d/slowonly_r50_8xb16-u48-240e_ntu60-xsub-keypoint.py --checkpoint https://download.openmmlab.com/mmaction/skeleton/posec3d/slowonly_r50_u48_240e_ntu60_xsub_keypoint/slowonly_r50_u48_240e_ntu60_xsub_keypoint-f3adabf1.pth --det-config demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py --det-checkpoint http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.9 --det-cat-id 0 --pose-config demo/demo_configs/td-hm_hrnet-w32_8xb64-210e_coco-256x192_infer.py --pose-checkpoint https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --label-map tools/data/skeleton/label_map_ntu60.txt
Loads checkpoint by http backend from path: http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth
Traceback (most recent call last):
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/datasets/base_det_dataset.py", line 40, in __init__
    super().__init__(*args, **kwargs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 247, in __init__
    self.pipeline = Compose(pipeline)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 37, in __init__
    transform = TRANSFORMS.build(transform)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 545, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 100, in build_from_cfg
    raise KeyError(
KeyError: 'mmdet.LoadImageFromFile is not in the transform registry. Please check whether the value of `mmdet.LoadImageFromFile` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "demo/demo_skeleton.py", line 188, in <module>
    main()
  File "demo/demo_skeleton.py", line 137, in main
    det_results, _ = detection_inference(args.det_config, args.det_checkpoint,
  File "/home/dl/openmmlab/mmaction2/mmaction/apis/inference.py", line 134, in detection_inference
    model = init_detector(
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/apis/inference.py", line 97, in init_detector
    metainfo = DATASETS.build(test_dataset_cfg).metainfo
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 545, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 135, in build_from_cfg
    raise type(e)(
KeyError: "class `CocoDataset` in mmdet/datasets/coco.py: 'mmdet.LoadImageFromFile is not in the transform registry. Please check whether the value of `mmdet.LoadImageFromFile` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'"

Please let me know how to fix it.

Suggest a potential alternative/fix

No response

Dai-Wenxun commented 1 year ago

Hi, @Ishihara-Masabumi Please update demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py according to https://github.com/open-mmlab/mmaction2/pull/2367. There is a bug, sorry.

Ishihara-Masabumi commented 1 year ago

Thank you for your reply. BTW, it made the same error, in spite of changing the config file as follows:

$ python3 demo/demo_skeleton.py demo/demo_skeleton.mp4 demo/demo_skeleton_out.mp4 --config demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py --checkpoint https://download.openmmlab.com/mmaction/skeleton/posec3d/slowonly_r50_u48_240e_ntu60_xsub_keypoint/slowonly_r50_u48_240e_ntu60_xsub_keypoint-f3adabf1.pth --det-config demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py --det-checkpoint http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.9 --det-cat-id 0 --pose-config demo/demo_configs/td-hm_hrnet-w32_8xb64-210e_coco-256x192_infer.py --pose-checkpoint https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --label-map tools/data/skeleton/label_map_ntu60.txt
Loads checkpoint by http backend from path: http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth
Traceback (most recent call last):
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/datasets/base_det_dataset.py", line 40, in __init__
    super().__init__(*args, **kwargs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 247, in __init__
    self.pipeline = Compose(pipeline)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 37, in __init__
    transform = TRANSFORMS.build(transform)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 545, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 100, in build_from_cfg
    raise KeyError(
KeyError: 'mmdet.LoadImageFromFile is not in the transform registry. Please check whether the value of `mmdet.LoadImageFromFile` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "demo/demo_skeleton.py", line 188, in <module>
    main()
  File "demo/demo_skeleton.py", line 137, in main
    det_results, _ = detection_inference(args.det_config, args.det_checkpoint,
  File "/home/dl/openmmlab/mmaction2/mmaction/apis/inference.py", line 134, in detection_inference
    model = init_detector(
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/apis/inference.py", line 97, in init_detector
    metainfo = DATASETS.build(test_dataset_cfg).metainfo
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 545, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 135, in build_from_cfg
    raise type(e)(
KeyError: "class `CocoDataset` in mmdet/datasets/coco.py: 'mmdet.LoadImageFromFile is not in the transform registry. Please check whether the value of `mmdet.LoadImageFromFile` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'"

Please let me know how to fix it, again.

Dai-Wenxun commented 1 year ago

What's the version of your mmdet? It works fine with mmdet 3.0.

Ishihara-Masabumi commented 1 year ago

mmdet version is 3.0.0.

Dai-Wenxun commented 1 year ago

Could you please show me the full content of demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py.

Ishihara-Masabumi commented 1 year ago

The full content of demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py is as follows:

# Copyright (c) OpenMMLab. All rights reserved.
# model settings
model = dict(
    type='FasterRCNN',
    _scope_='mmdet',
    data_preprocessor=dict(
        type='DetDataPreprocessor',
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        bgr_to_rgb=True,
        pad_size_divisor=32),
    backbone=dict(
        type='ResNet',
        depth=50,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        frozen_stages=1,
        norm_cfg=dict(type='BN', requires_grad=True),
        norm_eval=True,
        style='pytorch',
        init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')),
    neck=dict(
        type='FPN',
        in_channels=[256, 512, 1024, 2048],
        out_channels=256,
        num_outs=5),
    rpn_head=dict(
        type='RPNHead',
        in_channels=256,
        feat_channels=256,
        anchor_generator=dict(
            type='AnchorGenerator',
            scales=[8],
            ratios=[0.5, 1.0, 2.0],
            strides=[4, 8, 16, 32, 64]),
        bbox_coder=dict(
            type='DeltaXYWHBBoxCoder',
            target_means=[.0, .0, .0, .0],
            target_stds=[1.0, 1.0, 1.0, 1.0]),
        loss_cls=dict(
            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
        loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
    roi_head=dict(
        type='StandardRoIHead',
        bbox_roi_extractor=dict(
            type='SingleRoIExtractor',
            roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
            out_channels=256,
            featmap_strides=[4, 8, 16, 32]),
        bbox_head=dict(
            type='Shared2FCBBoxHead',
            in_channels=256,
            fc_out_channels=1024,
            roi_feat_size=7,
            num_classes=80,
            bbox_coder=dict(
                type='DeltaXYWHBBoxCoder',
                target_means=[0., 0., 0., 0.],
                target_stds=[0.1, 0.1, 0.2, 0.2]),
            reg_class_agnostic=False,
            loss_cls=dict(
                type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
            loss_bbox=dict(type='L1Loss', loss_weight=1.0))),
    # model training and testing settings
    train_cfg=dict(
        rpn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.7,
                neg_iou_thr=0.3,
                min_pos_iou=0.3,
                match_low_quality=True,
                ignore_iof_thr=-1),
            sampler=dict(
                type='RandomSampler',
                num=256,
                pos_fraction=0.5,
                neg_pos_ub=-1,
                add_gt_as_proposals=False),
            allowed_border=-1,
            pos_weight=-1,
            debug=False),
        rpn_proposal=dict(
            nms_pre=2000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0),
        rcnn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.5,
                neg_iou_thr=0.5,
                min_pos_iou=0.5,
                match_low_quality=False,
                ignore_iof_thr=-1),
            sampler=dict(
                type='RandomSampler',
                num=512,
                pos_fraction=0.25,
                neg_pos_ub=-1,
                add_gt_as_proposals=True),
            pos_weight=-1,
            debug=False)),
    test_cfg=dict(
        rpn=dict(
            nms_pre=1000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0),
        rcnn=dict(
            score_thr=0.05,
            nms=dict(type='nms', iou_threshold=0.5),
            max_per_img=100)))

# dataset settings
dataset_type = 'CocoDataset'
data_root = 'data/coco/'
file_client_args = dict(backend='disk')
test_pipeline = [
    dict(type='mmdet.LoadImageFromFile', file_client_args=file_client_args),
    dict(type='mmdet.Resize', scale=(1333, 800), keep_ratio=True),
    dict(
        type='mmdet.PackDetInputs',
        meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
                   'scale_factor'))
]
test_dataloader = dict(
    batch_size=1,
    num_workers=2,
    persistent_workers=True,
    drop_last=False,
    sampler=dict(type='DefaultSampler', shuffle=False),
    dataset=dict(
        type=dataset_type,
        data_root=data_root,
        ann_file='annotations/instances_val2017.json',
        data_prefix=dict(img='val2017/'),
        test_mode=True,
        pipeline=test_pipeline))

Dai-Wenxun commented 1 year ago

You didn't modify your config according to this PR https://github.com/open-mmlab/mmaction2/pull/2367, bro.

WILLOSCAR commented 1 year ago

You didn't modify your config according to this PR #2367, bro.

I have met this problem after modifying the config. env:

!pip install -U openmim
!mim install mmengine 'mmcv>=2.0.0rc1'  
!mim install "mmdet>=3.0.0rc5"
!mim install "mmpose>=1.0.0rc0"
mmengine version is 0.7.2

Ishihara-Masabumi commented 1 year ago

Thank you very much for your solution. But, there is another error as below.

$ python3 demo/demo_skeleton.py demo/demo_skeleton.mp4 demo/demo_skeleton_out.mp4 --config configs/skeleton/posec3d/slowonly_r50_8xb16-u48-240e_ntu60-xsub-keypoint.py --checkpoint https://download.openmmlab.com/mmaction/skeleton/posec3d/slowonly_r50_u48_240e_ntu60_xsub_keypoint/slowonly_r50_u48_240e_ntu60_xsub_keypoint-f3adabf1.pth --det-config demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py --det-checkpoint http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.9 --det-cat-id 0 --pose-config demo/demo_configs/td-hm_hrnet-w32_8xb64-210e_coco-256x192_infer.py --pose-checkpoint https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --label-map tools/data/skeleton/label_map_ntu60.txt
Loads checkpoint by http backend from path: http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth
Performing Human Detection for each frame
[                                                  ] 0/72, elapsed: 0s, ETA:04/12 11:25:49 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
04/12 11:25:49 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
Traceback (most recent call last):
  File "demo/demo_skeleton.py", line 188, in <module>
    main()
  File "demo/demo_skeleton.py", line 137, in main
    det_results, _ = detection_inference(args.det_config, args.det_checkpoint,
  File "/home/dl/openmmlab/mmaction2/mmaction/apis/inference.py", line 141, in detection_inference
    det_data_sample: DetDataSample = inference_detector(model, frame_path)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/apis/inference.py", line 177, in inference_detector
    results = model.test_step(data_)[0]
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
    return self._run_forward(data, mode='predict')  # type: ignore
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 326, in _run_forward
    results = self(**data, mode=mode)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 94, in forward
    return self.predict(inputs, data_samples)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/detectors/two_stage.py", line 227, in predict
    x = self.extract_feat(batch_inputs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/detectors/two_stage.py", line 110, in extract_feat
    x = self.backbone(batch_inputs)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/backbones/resnet.py", line 636, in forward
    x = self.conv1(x)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 457, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 453, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

Please let me know this solution.

zhangzihao666 commented 1 year ago

/home/user/anaconda3/envs/mmaction2/bin/python /home/user/mmaction2-main/demo/demo_skeleton.py Loads checkpoint by http backend from path: http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth Performing Human Detection for each frame [ ] 0/72, elapsed: 0s, ETA:04/12 10:59:56 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io 04/12 10:59:56 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future. [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 72/72, 3.2 task/s, elapsed: 23s, ETA: 0s Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth Performing Human Pose Estimation for each frame [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 72/72, 9.2 task/s, elapsed: 8s, ETA: 0s Loads checkpoint by http backend from path: https://download.openmmlab.com/mmaction/skeleton/posec3d/slowonly_r50_u48_240e_ntu60_xsub_keypoint/slowonly_r50_u48_240e_ntu60_xsub_keypoint-f3adabf1.pth 04/12 11:00:34 - mmengine - WARNING - Visualizer backend is not initialized because save_dir is None. Traceback (most recent call last): File "/home/user/mmaction2-main/demo/demo_skeleton.py", line 187, in main() File "/home/user/mmaction2-main/demo/demo_skeleton.py", line 181, in main visualize(args, frames, pose_data_samples, action_label) File "/home/user/mmaction2-main/demo/demo_skeleton.py", line 106, in visualize visualizer.add_datasample( File "/home/user/anaconda3/envs/mmaction2/lib/python3.8/site-packages/mmengine/dist/utils.py", line 360, in wrapper return func(*args, **kwargs) TypeError: add_datasample() got an unexpected keyword argument 'kpt_score_thr'

After modifying the farster-rcnn config, I also encountered the same problem and hope to receive your answer

Dai-Wenxun commented 1 year ago

The issue caused by kpt_score_thr has been solved at https://github.com/open-mmlab/mmaction2/pull/2381.

Dai-Wenxun commented 1 year ago

Guys can pull the latest main branch to update these modifications [https://github.com/open-mmlab/mmaction2/pull/2381 https://github.com/open-mmlab/mmaction2/pull/2380] to solve the issues mentioned above, thanks all for pointing these issues out.

zhangzihao666 commented 1 year ago

Guys can pull the latest main branch to update these modifications [https://github.com/[/pull/2381](https://github.com/open-mmlab/mmaction2/pull/2381) https://github.com/[/pull/2380](https://github.com/open-mmlab/mmaction2/pull/2380)] to solve the issues mentioned above, thanks all for pointing these issues out.

Thank you very much for your answer!

Ishihara-Masabumi commented 1 year ago

I tried to pull by the following manner.

$ git pull
Updating f05b1ddc..ad60cf03
Fast-forward
 .owners.yml                                                           |  16 ++++
 README.md                                                             | 143 +++++++++++++++----------------
 README_zh-CN.md                                                       | 135 +++++++++++++++--------------
 .../tsn_imagenet-pretrained-r50_8xb32-1x1x3-100e_kinetics400-rgb.py   |  15 +---
 demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py                |   2 +-
 demo/demo_skeleton.py                                                 |   2 +-
 docs/en/get_started/guide_to_framework.md                             |   2 +-
 docs/en/get_started/installation.md                                   |   1 -
 docs/en/get_started/overview.md                                       |   2 +-
 docs/en/index.rst                                                     |   2 +-
 docs/en/user_guides/{Inference.md => inference.md}                    |   0
 tests/models/utils/test_gradcam.py                                    |   1 +
 12 files changed, 162 insertions(+), 159 deletions(-)
 create mode 100644 .owners.yml
 rename docs/en/user_guides/{Inference.md => inference.md} (100%)

Then, I run the same command line above, but the error message is the same.

$ python3 demo/demo_skeleton.py demo/demo_skeleton.mp4 demo/demo_skeleton_out.mp4 --config configs/skeleton/posec3d/slowonly_r50_8xb16-u48-240e_ntu60-xsub-keypoint.py --checkpoint https://download.openmmlab.com/mmaction/skeleton/posec3d/slowonly_r50_u48_240e_ntu60_xsub_keypoint/slowonly_r50_u48_240e_ntu60_xsub_keypoint-f3adabf1.pth --det-config demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py --det-checkpoint http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.9 --det-cat-id 0 --pose-config demo/demo_configs/td-hm_hrnet-w32_8xb64-210e_coco-256x192_infer.py --pose-checkpoint https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --label-map tools/data/skeleton/label_map_ntu60.txt
Loads checkpoint by http backend from path: http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth
Performing Human Detection for each frame
[                                                  ] 0/72, elapsed: 0s, ETA:04/12 13:38:05 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
04/12 13:38:05 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
Traceback (most recent call last):
  File "demo/demo_skeleton.py", line 189, in <module>
    main()
  File "demo/demo_skeleton.py", line 139, in main
    det_results, _ = detection_inference(args.det_config, args.det_checkpoint,
  File "/home/dl/openmmlab/mmaction2/mmaction/apis/inference.py", line 144, in detection_inference
    det_data_sample: DetDataSample = inference_detector(model, frame_path)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/apis/inference.py", line 177, in inference_detector
    results = model.test_step(data_)[0]
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
    return self._run_forward(data, mode='predict')  # type: ignore
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 326, in _run_forward
    results = self(**data, mode=mode)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 94, in forward
    return self.predict(inputs, data_samples)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/detectors/two_stage.py", line 227, in predict
    x = self.extract_feat(batch_inputs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/detectors/two_stage.py", line 110, in extract_feat
    x = self.backbone(batch_inputs)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/models/backbones/resnet.py", line 636, in forward
    x = self.conv1(x)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 457, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 453, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

Please let me know how to fix it.

Dai-Wenxun commented 1 year ago

The "cuDNN error: CUDNN_STATUS_NOT_INITIALIZED" error usually occurs when there is an issue with the installation or initialization of the cuDNN library, which is a GPU-accelerated deep neural network library.

Here are some steps you can try to resolve this error:

Make sure that you have installed the latest version of cuDNN that is compatible with your CUDA version. You can download the cuDNN library from the NVIDIA website.

Check if your CUDA installation is properly configured. You can verify this by running a simple CUDA program.

Make sure that your GPU drivers are up to date. You can download and install the latest GPU drivers from the NVIDIA website.

If you are using a virtual environment, make sure that you have installed the necessary packages within that environment.

Try restarting your machine and then rerun your code to see if the issue persists.

If none of the above steps work, you can try uninstalling and reinstalling the cuDNN library.

If you are still encountering the same error, it may be helpful to provide more information about your setup, including the versions of cuDNN, CUDA, and your GPU drivers, as well as the code you are trying to run, so that the problem can be diagnosed more accurately.

Ishihara-Masabumi commented 1 year ago

Thanks. Which version do you recommend for cuDNN, CUDA?

Dai-Wenxun commented 1 year ago

Actually, I don't know the reason for your issue and it seems not the problem of mmaction2. Could you please find more information about your issue at here?

Ishihara-Masabumi commented 1 year ago

One of the test is below.

$ python3
Python 3.8.16 (default, Mar  2 2023, 03:21:46) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch.cuda
>>> torch.cuda.is_available()
True
>>> exit()

But, the other is below.

$ python3 test.py
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    output = m(input)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 307, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/dl/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 303, in _conv_forward
    return F.conv1d(input, weight, bias, self.stride,
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

Ishihara-Masabumi commented 1 year ago

Next, I reinstalled some library as follows:

pip3 install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html

Then, the uDNN error: CUDNN_STATUS_NOT_INITIALIZED disappeared. But, the next error occurred, as below.

$ python3 demo/demo_skeleton.py demo/demo_skeleton.mp4 demo/demo_skeleton_out.mp4 --config configs/skeleton/posec3d/slowonly_r50_8xb16-u48-240e_ntu60-xsub-keypoint.py --checkpoint https://download.openmmlab.com/mmaction/skeleton/posec3d/slowonly_r50_u48_240e_ntu60_xsub_keypoint/slowonly_r50_u48_240e_ntu60_xsub_keypoint-f3adabf1.pth --det-config demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py --det-checkpoint http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.9 --det-cat-id 0 --pose-config demo/demo_configs/td-hm_hrnet-w32_8xb64-210e_coco-256x192_infer.py --pose-checkpoint https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --label-map tools/data/skeleton/label_map_ntu60.txt
Traceback (most recent call last):
  File "/home/dl/openmmlab/mmaction2/mmaction/apis/inference.py", line 130, in detection_inference
    from mmdet.apis import inference_detector, init_detector
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/apis/__init__.py", line 2, in <module>
    from .det_inferencer import DetInferencer
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/apis/det_inferencer.py", line 21, in <module>
    from mmdet.evaluation import INSTANCE_OFFSET
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/evaluation/__init__.py", line 3, in <module>
    from .metrics import *  # noqa: F401,F403
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/evaluation/metrics/__init__.py", line 3, in <module>
    from .coco_metric import CocoMetric
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/evaluation/metrics/coco_metric.py", line 16, in <module>
    from mmdet.datasets.api_wrappers import COCO, COCOeval
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/datasets/__init__.py", line 14, in <module>
    from .utils import get_loading_pipeline
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/datasets/utils.py", line 5, in <module>
    from mmdet.datasets.transforms import LoadAnnotations, LoadPanopticAnnotations
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/datasets/transforms/__init__.py", line 6, in <module>
    from .formatting import ImageToTensor, PackDetInputs, ToTensor, Transpose
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/datasets/transforms/formatting.py", line 9, in <module>
    from mmdet.structures.bbox import BaseBoxes
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/structures/bbox/__init__.py", line 2, in <module>
    from .base_boxes import BaseBoxes
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/structures/bbox/base_boxes.py", line 9, in <module>
    from mmdet.structures.mask.structures import BitmapMasks, PolygonMasks
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/structures/mask/__init__.py", line 3, in <module>
    from .structures import (BaseInstanceMasks, BitmapMasks, PolygonMasks,
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmdet/structures/mask/structures.py", line 12, in <module>
    from mmcv.ops.roi_align import roi_align
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/ops/__init__.py", line 2, in <module>
    from .active_rotated_filter import active_rotated_filter
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/ops/active_rotated_filter.py", line 10, in <module>
    ext_module = ext_loader.load_ext(
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
    ext = importlib.import_module('mmcv.' + name)
  File "/home/dl/miniconda3/envs/openmmlab/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /home/dl/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNK2at10TensorBase8data_ptrIdEEPT_v

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "demo/demo_skeleton.py", line 189, in <module>
    main()
  File "demo/demo_skeleton.py", line 139, in main
    det_results, _ = detection_inference(args.det_config, args.det_checkpoint,
  File "/home/dl/openmmlab/mmaction2/mmaction/apis/inference.py", line 133, in detection_inference
    raise ImportError('Failed to import `inference_detector` and '
ImportError: Failed to import `inference_detector` and `init_detector` from `mmdet.apis`. These apis are required in this inference api!

This error seems not to be CUDNN error, but mmdet.apis error. Please let me know how to fix it.

Dai-Wenxun commented 1 year ago

Could you please check your versions of dependencies? https://mmaction2.readthedocs.io/en/latest/get_started/installation.html#troubleshooting

Ishihara-Masabumi commented 1 year ago

Thanks.