TMElyralab / MusePose

MusePose: a Pose-Driven Image-to-Video Framework for Virtual Human Generation
Other
1.85k stars 125 forks source link

Is there a way to use full face keypoints too? #37

Closed G-force78 closed 1 month ago

G-force78 commented 1 month ago

Hi, great work the consistency across frames is superb, I noticed you developed it partly from moore animate anyone, is there a way to use the face keypoints thay they use? https://github.com/MooreThreads/Moore-AnimateAnyone/blob/master/configs/inference/pose_videos/anyone-video-4_kps.mp4

TZYSJTU commented 1 month ago

Hi, great work the consistency across frames is superb, I noticed you developed it partly from moore animate anyone, is there a way to use the face keypoints thay they use? https://github.com/MooreThreads/Moore-AnimateAnyone/blob/master/configs/inference/pose_videos/anyone-video-4_kps.mp4

Yes, you can set draw_face = True in https://github.com/TMElyralab/MusePose/blob/main/pose/script/dwpose.py#L48 But the result will be wrong because we did not draw the face during training.

zhanghongyong123456 commented 1 month ago

Hi, great work the consistency across frames is superb, I noticed you developed it partly from moore animate anyone, is there a way to use the face keypoints thay they use? https://github.com/MooreThreads/Moore-AnimateAnyone/blob/master/configs/inference/pose_videos/anyone-video-4_kps.mp4

Yes, you can set draw_face = True in https://github.com/TMElyralab/MusePose/blob/main/pose/script/dwpose.py#L48 But the result will be wrong because we did not draw the face during training.

TZYSJTU

  1. yes,I test with face,Get blurry faces

https://github.com/TMElyralab/MusePose/assets/48466610/948ba738-1931-4220-9ffe-343efe248a2f

  1. I want to know how to remove the blue dot on finger image
TZYSJTU commented 1 month ago

Hi, great work the consistency across frames is superb, I noticed you developed it partly from moore animate anyone, is there a way to use the face keypoints thay they use? https://github.com/MooreThreads/Moore-AnimateAnyone/blob/master/configs/inference/pose_videos/anyone-video-4_kps.mp4

Yes, you can set draw_face = True in https://github.com/TMElyralab/MusePose/blob/main/pose/script/dwpose.py#L48 But the result will be wrong because we did not draw the face during training.

TZYSJTU

  1. yes,I test with face,Get blurry faces

https://github.com/TMElyralab/MusePose/assets/48466610/948ba738-1931-4220-9ffe-343efe248a2f 2. I want to know how to remove the blue dot on finger image

rgb <-> bgr. Just change the channel order of the img.

zhanghongyong123456 commented 1 month ago

Hi, great work the consistency across frames is superb, I noticed you developed it partly from moore animate anyone, is there a way to use the face keypoints thay they use? https://github.com/MooreThreads/Moore-AnimateAnyone/blob/master/configs/inference/pose_videos/anyone-video-4_kps.mp4

Yes, you can set draw_face = True in https://github.com/TMElyralab/MusePose/blob/main/pose/script/dwpose.py#L48 But the result will be wrong because we did not draw the face during training.

TZYSJTU

  1. yes,I test with face,Get blurry faces

https://github.com/TMElyralab/MusePose/assets/48466610/948ba738-1931-4220-9ffe-343efe248a2f 2. I want to know how to remove the blue dot on finger image

rgb <-> bgr. Just change the channel order of the img.

OK , about pose align video (About whether RGB or BGR format affects video generation?),For hand generation, rgb effect is not as good as bgr effect, is that true, or is it caused by my different alignment pose

TZYSJTU commented 1 month ago

Hi, great work the consistency across frames is superb, I noticed you developed it partly from moore animate anyone, is there a way to use the face keypoints thay they use? https://github.com/MooreThreads/Moore-AnimateAnyone/blob/master/configs/inference/pose_videos/anyone-video-4_kps.mp4

Yes, you can set draw_face = True in https://github.com/TMElyralab/MusePose/blob/main/pose/script/dwpose.py#L48 But the result will be wrong because we did not draw the face during training.

TZYSJTU

  1. yes,I test with face,Get blurry faces

https://github.com/TMElyralab/MusePose/assets/48466610/948ba738-1931-4220-9ffe-343efe248a2f 2. I want to know how to remove the blue dot on finger image

rgb <-> bgr. Just change the channel order of the img.

OK , about pose align video (About whether RGB or BGR format affects video generation?),For hand generation, rgb effect is not as good as bgr effect, is that true, or is it caused by my different alignment pose

During training, we use the red finger. So keeping it same when testing should be better.