search

jy0205 / Pyramid-Flow

Code of Pyramidal Flow Matching for Efficient Video Generative Modeling
https://pyramid-flow.github.io/
MIT License
2.03k stars 179 forks source link

Any options available for CPU offload for less VRAM? #46

Closed SoftologyPro closed 1 week ago

SoftologyPro commented 1 week ago

Using the default settings in the gradio app took 25 minutes on a 24 GB VRAM 4090 to create a movie. 24 GB was maxed out. Are there any cpu offload or other options available to reduce VRAM and speed up generation time?

"shrek eating pizza" using default settings https://github.com/user-attachments/assets/064ef30e-31b6-4c7c-93b2-791e097bd0c6

jy0205 commented 1 week ago

Thank you for your attention! Please modify the following two lines to enable cpu offloading: https://github.com/jy0205/Pyramid-Flow/blob/6eca678a40acac4c37e777ef9fd13a703850116a/app.py#L99 and https://github.com/jy0205/Pyramid-Flow/blob/6eca678a40acac4c37e777ef9fd13a703850116a/app.py#L121

On an 80GB VRAM A100, it takes approximately 6 minutes to generate a 5-second, 24 FPS, 768p video sample (with CPU offloading enabled). You might also consider reducing the sampling steps to save time by changing video_num_inference_steps to [8, 8, 8] (may affect the quality of the generated video)

feifeiobama commented 1 week ago

Closed since CPU offloading is supported. More CPU offloading features are on the way.