statho / ScoreHMR

ScoreHMR: Score-Guided Diffusion for 3D Human Recovery (CVPR 2024)
MIT License
367 stars 24 forks source link

Request for Guidance on Tracking Smoothing and Frame Completion for Constant Presence in Videos #19

Closed Sun-Happy-YKX closed 1 week ago

Sun-Happy-YKX commented 1 month ago

Thank you very much for your excellent work!

I am currently utilizing this reportire for a project involving video analysis where multiple people are present almost constantly throughout the video. However, I am encountering an issue where some individuals are not detected in certain frames despite their continuous presence in the scene. For example, in a sequence where I expect each person to be present across 300 frames, the detection results are varying (e.g., 300, 300, 287, 294, 276 frames respectively for each individual). I am seeking advice on how to handle these discrepancies with your tracking tools.

Could you please provide any suggestions on how I might modify the tracking logic to smooth out the presence and possibly fill in these missing frames? Specifically, I am looking to ensure that each person is accounted for in all 300 frames, whether this means interpolating missing data or adjusting the detection parameters.

statho commented 1 month ago

Thanks for appreciating our work! We are running ScoreHMR in top of PHALP tracklets, which means that means that we rely on the identification made by PHALP for each human. What we can do is interpolate the prediction for frames that humans are occluded (see here for example), when PHALP does not lose track of their ids in the following frames.

To improve the tracking on your specific use case, you would need to adjust the parameters of the PHALP tracker. Please have a look at the PHALP codebase and contact the authors if needed.