search

open-mmlab / mmpose

OpenMMLab Pose Estimation Toolbox and Benchmark.
https://mmpose.readthedocs.io/en/latest/
Apache License 2.0
5.81k stars 1.24k forks source link

Roadmap of MMPose #9

Open hellock opened 4 years ago

hellock commented 4 years ago

We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan is also available here.

You can either:

  1. Suggest a new feature by leaving a comment.
  2. Vote for a feature request with πŸ‘ or be against with πŸ‘Ž. (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!)
  3. Tell us that you would like to help implement one of the features in the list or review the PRs. (This is the greatest things to hear about!)
aa12356jm commented 4 years ago

i hope that MMPose can support 21 hand landmark detetion, thanks

jin-s13 commented 4 years ago

i hope that MMPose can support 21 hand landmark detetion, thanks

Good suggestions! We will add this feature in our TODO list. Thank you.

jin-s13 commented 4 years ago

TODO List (continuously updated... [last edit: 2023.1.14]) : Here is a collection of feature requests. Items that have already been implemented in MMPose will be removed from the list.

  1. More popular backbones

  2. Add more popular datasets:

  3. More 2d human pose estimation method.

  4. More 2d face alignment algorithms.

  5. More 3d human pose algorithms.

  6. Support 2d video pose estimation and tracking

  7. Support Vehicle pose estimation

  8. Add 3D Pose Consistency Benchmark #828

  9. Mano based hand keypoints detection

  10. Depth-based 3d hand pose estimation

  11. Multi-view 3d pose estimation

  12. Support memonger

  13. Support Pytorch AMP training #339

  14. Hyperparameter tuner Optuna

  15. Support Unity plugin

  16. print loss during evaluation. #333

  17. Quantization Aware Training #359

  18. Easier Usage (API)

  19. Export to Torchscript #576

gachiemchiep commented 4 years ago

Would you mind add https://github.com/open-mmlab/mmpose/issues/31#issuecomment-663334223 to the TODO list.

jin-s13 commented 4 years ago

Would you mind add #31 (comment) to the TODO list.

Sure.

innerlee commented 4 years ago

Speed up inference https://github.com/open-mmlab/mmpose/issues/40

jin-s13 commented 4 years ago

Support video pose estimation #67

pablovela5620 commented 4 years ago

Would be great to add support for whole body pose estimation dataset (body+face+hands) via COCO-WholeBody

lisa676 commented 4 years ago

Also add support for MPII in mmdetection.

OasisYang commented 4 years ago

It would be great to add support for pose tracking dataset i.e. posetrack2017/2018.

flynnamy commented 4 years ago

Support to convert pytorch model to onnx by the way.Thx！

innerlee commented 4 years ago

@OasisYang could you elaborate? is loading the data and processing on a frame basis is enough, or you want the tracking part also?

@flynnamy sounds like a request for a general tool. maybe we can provide such tools for the whole mm-series (just saying, not a confirmation).

OasisYang commented 4 years ago

@innerlee If possible, adding both data loading and tracking part would be great. However, the tracking part seems a little bit complicate and always comes with some extra modules. Maybe, the first step is basically to support the data loading and processing. Thanks

@OasisYang could you elaborate? is loading the data and processing on a frame basis is enough, or you want the tracking part also?

@flynnamy sounds like a request for a general tool. maybe we can provide such tools for the whole mm-series (just saying, not a confirmation).

jin-s13 commented 4 years ago

Support ShuffleNet V2 & MobileNet V3 backbones. #94

pablovela5620 commented 4 years ago

Support for InterHand2.6

Witek- commented 4 years ago

Add Yolov4 and OpenPose

fhir-obesity commented 4 years ago

Please make it possible to obtain estimated heatmaps from methods

innerlee commented 4 years ago

@hamedcan could you explain more about the usage? do you want a visualizer of heatmaps during training, or a visualization tool for demo, or anything else?

san-guy commented 4 years ago

Bottom up for MPII dataset?

fhir-obesity commented 4 years ago

@innerlee First, I really want to thank you for the MMPose. It really helped me. I want to compare different models' performance on hard poses. So I need to be able to observe generated heatmaps. I want a visualization tool for demo.

HuangJunJie2017 commented 4 years ago

support memonger : https://github.com/Lyken17/pytorch-memonger

jin-s13 commented 4 years ago

Support multi-head networks #219

flynnamy commented 4 years ago

Please support mpii_trb demo and mpi_inf_3dhp datasets!

yanglilian commented 3 years ago

Support 3d hand keypoint estimation!!!!!!

yulong314 commented 3 years ago

Support log info when dataset is tinty, #333

YibinXie commented 3 years ago

Support PyTorch AMP training, thanks. #339

YibinXie commented 3 years ago

Support GCN-based methods for refining top-down results. https://arxiv.org/pdf/2003.10506v3.pdf https://github.com/lingtengqiu/OPEC-Net https://arxiv.org/abs/2007.10599

pablovela5620 commented 3 years ago

Would be great to see the integration of a hyperparameter tuner like Optuna

MaxGodTier commented 3 years ago

A Unity plugin would be amazing to have, using json input data and/or real-time pose estimation with a webcam and seeing it reflected on a 3D model.

innerlee commented 3 years ago

@MaxGodTier do you have experience in developing unity plugin? contributions are welcome :D

MaxGodTier commented 3 years ago

I don't, but a dirty implementation may be possible using an existing repo , it reads pose data from simple text files each representing a single frame , I see two solutions: (1) If pose_results from mmpose were translated into the same format expected from that repo, it will work out of the box without needing to change a single line of code or (2) edit the repo code (C#) to use mmpose rules instead of theirs.

aa12356jm commented 3 years ago

Quantization Aware Training for models to get the int8 models ,int8 models will greatly improve inference speed #359,thanks

zgs731 commented 3 years ago

Support DetTrack and KeyTrack. http://arxiv.org/abs/2003.13743 & https://arxiv.org/abs/1912.02323

pablovela5620 commented 3 years ago

Albumentations augmetnations similar to mmclassification

aa12356jm commented 3 years ago

i hope that MMPose can support 3D hand landmarks detetion, thanks

rhiver commented 3 years ago

Does MMPose support Single Person Pose Estimation? Currently I found only multi-person versions are supported.

innerlee commented 3 years ago

@rhiver single is a case of multi

rhiver commented 3 years ago

@rhiver single is a case of multi

Sort of. But multi-person version has two stages, person detection and pose estimation, which have to infer on two models. So this method doesn't work for realtime pose estimation in mobile devices since it takes too long on the inference. MobileNetV2 is good enough for simple pose estimation. But for best FPS, it's better to let it do both single person detection and pose estimation.

flynnamy commented 3 years ago

I see it has supported heatmap method of face datasets now. Please support regression method of face dataset!

jin-s13 commented 3 years ago

I see it has supported heatmap method of face datasets now. Please support regression method of face dataset!

Do you have any recommended papers/codes ?

flynnamy commented 3 years ago

I see it has supported heatmap method of face datasets now. Please support regression method of face dataset!

Do you have any recommended papers/codes ?

Yes,wingloss:https://arxiv.org/pdf/1711.06753.pdf, and GCN+softwing loss: https://arxiv.org/pdf/2006.11697.pdf

innerlee commented 3 years ago

Support FashionAI https://tianchi.aliyun.com/competition/entrance/231648/introduction

CHRIBUR0309 commented 3 years ago

I want to use mmpose with the pypi package much more easily than now; such as:

from mmpose import top_down

top_down("darkpose", "COCO_wholebody", video_path="hoge.mp4", output_json_title="hoge") # Analyze hoge.mp4 with COCO wholebody on darkpose and output the result as hoge/hoge000000000000.json, hoge/hoge000000000001.json, hoge/hoge000000000002.json, ....
yeshaokai commented 3 years ago

MPII multi-person dataset for bottom-up methods is really needed!

aa12356jm commented 3 years ago

hi ,can you add handlandmark filtering algorithm for eliminating handlandmark jittering in videos? thanks

dineshreddy91 commented 3 years ago

Adding Vehicle pose estimation to the pipe line using CarFusion dataset. Similar to Occlusion-net, and Apollocar3D.

simba0703 commented 3 years ago

Export to Torchscript !

pablovela5620 commented 3 years ago

Lite-HRNet, its already built with mmpose, so including into the main repo should be super simple. Would be amazing if it could work with the pytorch2onnx tool for deployment

ioctl-user commented 3 years ago

Please support Halpe data set: https://github.com/Fang-Haoshu/Halpe-FullBody

It has 3 useful points in addition to the COCO-WholeBody.

rubeea commented 3 years ago

Hi everyone,

I intend to create my own keypoints dataset with 3 points of interest (two endpoints and one center point). Can anyone kindly help me on how I can create annotations to be loaded into mmpose? Because I believe that the repo is based on mmcv, how can I get my own dataloader? Any help in this regard will be highly appreciated. Thank you