Can MusePose directly use human Pose video as input?

[FangSen9000]

Can MusePose directly use human Pose video as input? Or does the model have to acquire the hidden posture information itself as an internal control condition? @jhj0517 @erjanmx

[jhj0517]

If you have already extracted the pose from the video, you don't need to do it again.

The processes are:

1. Extract the pose from both the video and the image, and resize the video pose to match the image pose
2. Apply the resized pose video to the image (using other models)

So, if you have already completed step 1, you don't need to do it again.

But it is recommended to do step 1 each time you run inference because the video pose is resized to match the image pose in step 1.

You can try the HuggingFace demo here to see how the process works: https://huggingface.co/spaces/jhj0517/musepose