search

damo-cv / RealisDance

The official implementation of RealisDance
Apache License 2.0
149 stars 8 forks source link

stage1 for a customized video #4

Open cmmclee opened 3 weeks ago

cmmclee commented 3 weeks ago

Hello, the effects in the demo look great. I have a question, how to get such parameters from a customized video? --smpl __assets__/demo_seq/smpl_1.mp4 --hamer __assets__/demo_seq/hamer_1.mp4 --dwpose _assets__/demo_seq/dwpose_1.pkl

theFoxofSky commented 3 weeks ago

We get the pose sequences using the following methods: SMPL: https://github.com/caizhongang/SMPLer-X HAMER: https://github.com/geopavlakos/hamer DWPose: https://github.com/IDEA-Research/DWPose

Note that we rewrote the rendering code for SMPL and HAMER, making the pose colorful. But if you want to train your own RealisDance, you can just use the standard SMPL and HAMER (without color).

theFoxofSky commented 3 weeks ago

We will also release the rendering codes, but this may take a while as they are not cleaned up yet.

cmmclee commented 3 weeks ago

Thank you very much for your quick reply. In addition, what I want to know is, if I only use ‘standard SMPL and HAMER’, can I get as good an effect as in the demo?

theFoxofSky commented 3 weeks ago

I have not tried this on the ckpt we released. However, I tried training an earlier version of the RealisDance model using standard SMPL and HAMER, which achieved similar results for most cases when using longer training steps.

yuchenli-sony commented 6 days ago

great THANK U FOR UR EXCELLENT WORK