search

MiracleDance / PoseRAC

PoseRAC: Pose Saliency Transformer for Repetitive Action Counting
MIT License
13 stars 2 forks source link

Counting in real scenarios #2

Closed yinntag closed 1 year ago

yinntag commented 1 year ago

I have achieved remarkable counting results by using the pre-trained model you provided and praise it for a good job, but I think there are also two issues to consider.

(1) Even counting in real scenarios seems to need to use BlazePose to pre-extract key points as input. Can this process be simplified?

(2) How to modify or redesign a more general counting model when multiple repetitive actions are performed simultaneously?

Many thanks for your help.

MiracleDance commented 1 year ago

Thanks for your issues.

(1) In our PoseRAC framework, we count the repetitive actions based on the poses, so pose is essential and important information, which can not be removed. Counting in real scenarios needs real-time pose estimation, while BlazePose is a lightweight architecture designed for mobile devices, so the whole process will not have too much time burden. Or a faster pose estimation model instead of BlazePose?

(2) Counting multiple repetitive actions simultaneously may not be realized directly by our PoseRAC, but it can upgrade. The possible ideas include: i) Replace BlazePose with other networks that can estimate multiple poses. ii) How to use object tracking to determine pose chain for each repetitive action?