search

HVision-NKU / StoryDiffusion

Accepted as [NeurIPS 2024] Spotlight Presentation Paper
Apache License 2.0
5.83k stars 581 forks source link

How much compute for training the motion predictor? #6

Closed olliacc closed 5 months ago

zhoudaquan commented 5 months ago

Hi,

The motion predictor used 8 GPU cards for 3-4 days. In addition, we also train our self-owned motion module which uses 32 A100 GPU cards for one week. The motion predictor and the motion module is used together for the video result’s presented in the paper.

zhoudaquan commented 5 months ago

The main idea behind this way of video generation is to save the compute since image model is much more easier to train, compared to train a hyper-realistic video model directly.