search

ViTAE-Transformer / ViTPose

The official repo for [NeurIPS'22] "ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation" and [TPAMI'23] "ViTPose++: Vision Transformer for Generic Body Pose Estimation"
Apache License 2.0
1.37k stars 186 forks source link

Running video demo #20

Open rmhiri-gpsw opened 2 years ago

rmhiri-gpsw commented 2 years ago

Hello,

I tried to run the video demo using mmdet:

python demo/top_down_pose_tracking_demo_with_mmdet.py ./demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py ./faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_base_coco_256x192.py ./vitpose-b.pt --video-path ./test.MOV --out-video-root ./output_video/ but I have errors due to the version compatibility between mmcv, mmdet et the current ViTPose (or mmpose) version.

So here what I do, I install mmcv from sources (1.3.9 version as recommended in the read me of this repo) and the mmdet from sources as well (I tried last mmdet, mmdet==2.14.0 as it is recommended in the mminstall.txt for mmpose 0.24.0: ['mmcv-full>=1.3.8', 'mmdet>=2.14.0', 'mmtrack>=0.6.0']., and mmdet==2.23.0)

here is what I have with the following versions for example (pip list) : mmcv 1.3.9
mmdet 2.14.0
mmpose 0.24.0

Note that I use this : torch 1.11.0+cu113 torchvision 0.12.0+cu113

I got this error : /home/ubuntu/venv/lib/python3.8/site-packages/mmcv/cnn/bricks/transformer.py:27: UserWarning: Fail to import ``MultiScaleDeformableAttention`` from ``mmcv.ops.multi_scale_deform_attn``, You should install ``mmcv-full`` if you need this module. warnings.warn('Fail to import ``MultiScaleDeformableAttention`` from ' Traceback (most recent call last): File "demo/top_down_pose_tracking_demo_with_mmdet.py", line 190, in <module> main() File "demo/top_down_pose_tracking_demo_with_mmdet.py", line 74, in main assert has_mmdet, 'Please install mmdet to run the demo.' AssertionError: Please install mmdet to run the demo.

when I put mmdet to 2.23.0 I got this error :

AssertionError: MMCV==1.3.9 is used but incompatible. Please install mmcv>=1.3.17, <=1.6.0.

Tried to set mmcv>=1.3.17, did not resolve the problem !

can you please tell us which versions (mmcv and mmdet) are recommended to run ViTPose on videos ?

Annbless commented 2 years ago

Hi,

you can try the web demo here link.

As demonstrated in the error msg, it seems that the error is caused that the installed package is mmcv not mmcv-full. Please use MMCV_WITH_OPS=1 pip install xxx to install mmcv-full.

kbpk commented 2 years ago

Hi,

it worked for me like in the web demo with YOLOX (proper installation order):

YuBeomGon commented 2 years ago

thank kbpk, I get great help

for nvidia rtx 2080ti , 3090ti its working

install cuda 11.7 (compatible to torch 1.12.1) https://developer.nvidia.com/cuda-11-7-1-download-archive?target_os=Linux&target_arch=x[…]ribution=Ubuntu&target_version=20.04&target_type=deb_local rm -rf ~/anaconda3/envs/mmdemo conda create --name mmdemo python=3.9 -y conda activate mmdemo

  1. install torch 1.12.1 https://pytorch.org/get-started/previous-versions/ conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge
  2. mmcv (it takes long time) (need to mmcv>=1.3.17, <=1.7.0.) pip install mmcv-full==1.5.0 -f https://download.openmmlab.com/mmcv/dist/cu117/torch1.12.1/index.html https://github.com/open-mmlab/mmcv ViTPose https://github.com/ViTAE-Transformer/ViTPose

git clone https://github.com/ViTAE-Transformer/ViTPose.git cd ViTPose pip install -v -e . pip install timm==0.4.9 einops mmdet 3rd party package install pip install mmdet https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md/#Installation pose inference with det (in ViTPose ) python demo/top_down_video_demo_with_mmdet.py demo/mmdetection_cfg/retinanet_swin-t-p4-w7_fpn_2x_coco.py demo/mmdet_checkpoints/swin_t_p4_w7_naju_epoch_24.pth configs/inter/ViTPose_base_coco_256x192.py work_dirs/ViTPose_base_coco_256x192/best_AP_epoch_70.pth --video-path video/out_7.mp4 --out-video-root outputs (편집됨)

ZhengdiYu commented 1 year ago

thank kbpk, I get great help

for nvidia rtx 2080ti , 3090ti its working

install cuda 11.7 (compatible to torch 1.12.1) https://developer.nvidia.com/cuda-11-7-1-download-archive?target_os=Linux&target_arch=x[…]ribution=Ubuntu&target_version=20.04&target_type=deb_local rm -rf ~/anaconda3/envs/mmdemo conda create --name mmdemo python=3.9 -y conda activate mmdemo 2. install torch 1.12.1 https://pytorch.org/get-started/previous-versions/ conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge 3. mmcv (it takes long time) (need to mmcv>=1.3.17, <=1.7.0.) pip install mmcv-full==1.5.0 -f https://download.openmmlab.com/mmcv/dist/cu117/torch1.12.1/index.html https://github.com/open-mmlab/mmcv ViTPose https://github.com/ViTAE-Transformer/ViTPose

git clone https://github.com/ViTAE-Transformer/ViTPose.git cd ViTPose pip install -v -e . pip install timm==0.4.9 einops mmdet 3rd party package install pip install mmdet https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md/#Installation pose inference with det (in ViTPose ) python demo/top_down_video_demo_with_mmdet.py demo/mmdetection_cfg/retinanet_swin-t-p4-w7_fpn_2x_coco.py demo/mmdet_checkpoints/swin_t_p4_w7_naju_epoch_24.pth configs/inter/ViTPose_base_coco_256x192.py work_dirs/ViTPose_base_coco_256x192/best_AP_epoch_70.pth --video-path video/out_7.mp4 --out-video-root outputs (편집됨)

It's still not working for me. It seems the mmcv version needed for mmdet is always not compatible the version that mmpose used in this repo needs.