search

Boese0601 / MagicDance

[ICML 2024] MagicPose(also known as MagicDance): Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion
https://boese0601.github.io/magicdance/
Other
629 stars 52 forks source link

How can I train my own video data? Great job, looking forward to your reply! #10

Closed Jeff-Fudan closed 5 months ago

Boese0601 commented 5 months ago

For training on your own dataset, you first need to run openpose for your input images/videos and save the visualized pose map. Then, organize them as the format shown in the TikTok dataset. You can also refer to ControlNet-OpenPose, we use exactly the same Pose ControlNet in our pipeline. Then set the path to your data in dataset/tiktok_video_arnold_copy.py

Your Dataset
|----train_set
  |----video_000
    |----000.jpg
    |----001.jpg
    |----002.jpg
    ...
  |----video_001
  |----video_002
  ...
|----pose_map_train_set
  |----video_000
    |----000.jpg
    |----001.jpg
    |----002.jpg
    ...
  |----video_001
  |----video_002
  ...
|----val_set
|----pose_map_val_set
|----test_set
|----pose_map_test_set
|----...