search

Walter0807 / MotionBERT

[ICCV 2023] PyTorch Implementation of "MotionBERT: A Unified Perspective on Learning Human Motion Representations"
Apache License 2.0
1.06k stars 131 forks source link

Some sport demo relative to different clip length #58

Closed valentin-fngr closed 1 year ago

valentin-fngr commented 1 year ago

Hi,

Here are some 3D skeletons I extracted from sport clips (tennis).

Tennis back hand : X3D (4) (1)

A longer tennis sequence : X3D (11)

3D representation out of an 8 min vieos best_federer (1)

I have noticed that the longer the sequence, the better the 3D pose estimation is. I have also noticed that for a same clip, the angle of the 3D visualization changes with respect to the clip length.

I observe some jittering for very short clips. If you guys have any idea how I can reduce that effect, please let me know.

I have a script extracting the 2D skeleton using AlphaPose and then generating the 3D poses. My 2D pose estimation is very good, but I still get this jittering effect.

Walter0807 commented 1 year ago

Hi, could you please provide more information (model, config, window size) for the results? It seems like fig 1,2,3 are not from the same model (1,2 root fixed, 3 not).

Walter0807 commented 1 year ago

Intuitively, a longer clip length would give better results, which is also verified on Human 3.6M.

valentin-fngr commented 1 year ago

Hi, could you please provide more information (model, config, window size) for the results? It seems like fig 1,2,3 are not from the same model (1,2 root fixed, 3 not).

Hi, thanks for your reply. You are right, GIF 1 and 2 are coming from the same "setup" while GIF 3 has been predicted with another training setup. Unfortunately, I do not remember the modification I did. Let me investigate and get back to you.

Is there any reason GIF 1 and 2 would have a fixed root and not 3 ?

Walter0807 commented 1 year ago

Hi, could you please provide more information (model, config, window size) for the results? It seems like fig 1,2,3 are not from the same model (1,2 root fixed, 3 not).

Hi, thanks for your reply. You are right, GIF 1 and 2 are coming from the same "setup" while GIF 3 has been predicted with another training setup. Unfortunately, I do not remember the modification I did. Let me investigate and get back to you.

Is there any reason GIF 1 and 2 would have a fixed root and not 3 ?

It's related to the rootrel flag in configs.

valentin-fngr commented 1 year ago

Hi,

Did you update the pose3D weights for the global_lite model recently ? I had two version of this repo, one older one more recent.

I had to copy the weights from the old repo to the new one to make it work on short clips :

Description of the GIF

Important note : I have tried the two weights with the same alphapose-results.json to make sure I am testing in the same conditions for both weights.

Walter0807 commented 1 year ago

No it has not been changed.Best,WentaoSent from my iPhoneOn 2 Jul 2023, at 06:43, valentin-fngr @.***> wrote: Hi, Did you update the pose3D weights for the global_lite model recently ? I had two version of this repo, one older one more recent. I had to copy the weights from the old repo to the new one to make it work on short clips :

Important note : I have tried the two weights with the same alphapose-results.json to make sure I am testing in the same conditions for both weights.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>