mmpose是否支持使用摄像头实时检测呢？

[ShaneCan]

Ccould I use the camera to detect stuffs by mmpose in real time ？mmpose是否支持使用摄像头进行实时检测呢？

[ly015]

Yes, We recommend using the pose estimation demo developed with MMPose WebcamAPI, which is a handy framework to develop simple demo/applications with camera inputs.

Alternatively, you can also try the old webcam demo

[ShaneCan]

Yes, We recommend using the pose estimation demo developed with MMPose WebcamAPI, which is a handy framework to develop simple demo/applications with camera inputs.

Alternatively, you can also try the old webcam demo

thx ly015, but when i run this demo, i have FileNotFoundError: file "C:\Users\DELL\PycharmProjects\mmpose-0.24.0\demo\demo\mmdetection_cfg\ssdlite_mobilenetv2_scratch_600e_coco.py" does not exist... idk whats this py file

[ly015]

I guess it's because you run demo/webcam_demo.py from the demo/ folder. Please note that both demo/webam_demo.py and tools/webcam/run_webcam.py should be launched from the mmpose root path.

[ShaneCan]

I guess it's because you run demo/webcam_demo.py from the demo/ folder. Please note that both demo/webam_demo.py and tools/webcam/run_webcam.py should be launched from the mmpose root path.

got it ! i really appreciate ur patient to answer my question, and i m curious about could i use mmpose to detect two or three different type of object's keypoint in the same time? and use mmdet api to detect what the type of object,i mean every different objects with different keypoints ... if mmpose suppose that, when i perpare the datesets,should i need to add all the keypoints in one pic and give a rectangle frame？how should i make datasets, thx again!!!

[ShaneCan]

I guess it's because you run demo/webcam_demo.py from the demo/ folder. Please note that both demo/webam_demo.py and tools/webcam/run_webcam.py should be launched from the mmpose root path.

i saw in webcam_demo.py , there are two different object's checkpoint,which is person and cat. is that mean when i train a model, i should separate different objects in different pic and independent labeling and training to get different train model for each object? could i add 3 or 4 different objects to detect in this demo? really apreciate!!

[jin-s13]

To detect the pose of different objects, you may 1) Train separate pose models for each object (person and cat) 2) Train a single pose model, and separate different keypoints in different out channels, e.g the first 17 keypoints for human pose and the remaining keypoints for cat pose.

[ShaneCan]

To detect the pose of different objects, you may

1. Train separate pose models for each object (person and cat)
2. Train a single pose model, and separate different keypoints in different out channels, e.g the first 17 keypoints for human pose and the remaining keypoints for cat pose.

gotcha, i perfer first one......and could i add 3 or 4 different objects to detect in this demo? and i found the pose detect must after mmdet detect, if mmdet couldnt detect that object, then mmpose wont detect, is that true? if i close mmdet, is mmpose still running after mmdet detect? mmpose必须要在mmdet识别该物体后才会识别姿态吗？ 十分感谢二位的耐心解答！！best wishes for mmpose!!!

[ly015]

Pose estimation methods can be divided into two major categories: top-down and bottom-up. Top-down methods use the bounding box area of an object as the input, and output pose estimation of a single target, thus it needs a detection model to obtain the bbox at first. Bottom-up methods directly detect all keypoints (identity agnostic) from the whole image and then group the keypoints and assign them to different targets.

We use top-down models in our webcam demo, that's why there is a detection model (supported by MMDetection) followed by multiple mmpose models. If you are interested in bottom-up methods, please check out our bottom-up demo.

[hust-kevin]

@ly015 @jin-s13 hi, 2d-to-3d lifting video-based method like videopose3d, only can use for offline video? If I want detect 3d pose on live stream, we should use simplebaseline rather than videopose3d? what' more, do you konw some 3d pose detection methed can use for live stream in recent year?

[jin-s13]

I think 2d-to-3d lifting is efficient enough. And it can run realtime.

[hust-kevin]

@jin-s13 videopose3d process frame-i, need use frame between (i - K, i + k), but in the live strem, we can't get frame after frame i, so this method can't use for imge or live stream?

what's more, in body3d_two_stage_img_demo.py, we need pass the camera-param, but in the wild image, we don't know the camera-param, how can we use the demo predict 3d pose?

[jin-s13]

Please check
causal (bool): Use causal convolutions instead of symmetric convolutions (for real-time applications). Default: False.

Using the default camera-param is ok.

@ly015 Do you have any suggestions?

[hust-kevin]

@jin-s13 according to the demo.md，use the tests/data/h36m/cameras.pkl, if we not use the camera-param, the vis result likee this

when infer in the wild image, we don't have camera-param, you sad Using the default camera-param is ok. so , just use the tests/data/h36m/cameras.pkl?

[ShaneCan]

could i though adding some codes like ''parser.add_argument( '--object3-pose-config',--object3-pose-checkpoint',--object3-det-ids') and if args.enable_object3_pose '' to detect more than two objects?

[ly015]

@hust-kevin Yes, that is the default camera parameter file which is also used in model training. Usually, you can get visually reasonable results using the default camera parameters, but the results are not likely to be very accurate and the performance may highly depend on factors like image resolution, human posture, and background, et al.

[ly015]

@ShaneCan Please checkout https://github.com/open-mmlab/mmpose/blob/master/demo/docs/3d_human_pose_demo.md#3d-human-pose-two-stage-estimation-video-demo for multi-person 3D keypoint detection.

[exceedTu]

摄像头实时检测是不是只能在linux环境实现，Windows下只能推理照片或者视频 ？

[ly015]

@exceedTu MMPose 的 webcam demo 和 webcam api 是可以再 windows 下使用的

[exceedTu]

怎么才能做到像paddle那样的 直接运行一个 inference.py 就能直接显示推理窗口，我运行了： python demo/webcam_demo.py --config demo/webcam_cfg/pose_estimation.py python demo/webcam_demo.py --config demo/webcam_cfg/pose_tracking.py python demo/webcam_demo.py --config demo/webcam_cfg/gesture_recognition.py

这三个命令，发现效果都很差。

还有我只想单独进行人体关键点检测的话，应该在哪里修改配置。

期待回复，感谢。

[Ben-Louis]

@exceedTu 您好，关于您的几个问题：

1. 您提到的三个命令都是可以直接显示推理窗口的。如果想要直接运行一个 inference.py 就显示推理窗口，您可以做进一步封装
2. 请问效果差主要是指精度还是速度呢？
   • 如果是精度问题，在您算力有富余的情况下，可以将 webcam config 里面 TopDownPoseEstimatorNode 的模型换成精度更高的模型，具体可参考模型库
   • 如果是速度问题，可以尝试去掉用不到的 node, 参考3
3. 只想单独进行人体关键点检测，需要对 demo/webcam_cfg/pose_estimation.py 进行以下修改：
   • 删除 动物姿态估计的 node
   • 将 object assigner node 中的 object_buffer 设为 human_pose

[exceedTu]

收到，我接着试试，谢谢