search

DeepMotionEditing / deep-motion-editing

An end-to-end library for editing and rendering motion of 3D characters with deep learning [SIGGRAPH 2020]
BSD 2-Clause "Simplified" License
1.55k stars 254 forks source link

About the 2d pose extraction in Openpose #169

Open shinshiner opened 3 years ago

shinshiner commented 3 years ago

Hi, thanks for sharing your great work! I'm curious about the style transfer in 2D space and want to learn more about it. Could you share the 2d pose extraction settings? e.g. video resolution, parameters when running openpose. BTW, can this model be applied to any videos? I've not seen the processing for scale adaption, and the results seem to be different when giving two videos with different resolutions ?

shinshiner commented 3 years ago

Or could you provide some videos used in your paper and demo video for reference?

shinshiner commented 3 years ago

As mentioned in previous issues, you extract the normalization parameters from a video. When I apply some transforms, e.g. scaling, rotation, the model cannot generate reasonable results since the distribution of motion has changed. If I normalize the input motion indepentently, there is a relatively huge gap between the results and your demo video.

The question is how to enhance the scalability for video input, or I need to re-train the model if I want to use another video set to give style information?