search

cyber-meow / anime_screenshot_pipeline

A 99% automatized pipeline to construct training set from anime and more for text-to-image model training
MIT License
178 stars 10 forks source link

Add support for creating motion module datasets #28

Open FizzleDorf opened 8 months ago

FizzleDorf commented 8 months ago

Add option for creating small video clips from video. This is for any animate diff repo (training scripts are included with repos) or the motion module training gui. This paper also requires video and images for it's method so doing both image and video datasets at the same time may be something to consider for the future. ffmpeg is already a requirement so this should be possible.

cyber-meow commented 8 months ago

Thank you for the suggestion. I may look into it when I have time. However I think this will require some work as we either need dedicated tools for video clips or find a way to combine information from different frames to form the prompt. I will already need to try training with video clips myself to see what kind of dataset is meaningful in this case.