sample.py uses 45GB GPU RAM

Bujiazi / MotionClone

Official implementation of MotionClone: Training-Free Motion Cloning for Controllable Video Generation

290 stars 14 forks source link

sample.py uses 45GB GPU RAM #5

Closed askerlee closed 2 weeks ago

askerlee commented 2 weeks ago

First thanks for the amazing model.

When I ran it I noticed it took 45 GB of GPU RAM. Fortunately my GPU has a total RAM of 48GB. But I'm worried that this may prevent many people from using this model.

May I know what's the reason so much RAM is used? Is this because all the frames are processed at the same time? Any rough idea how to cut it down? Thanks.

Bujiazi commented 2 weeks ago

Thank you for your feedback. Currently, MotionClone utilizes higher GPU memory due to the following reasons:

We have introduced classifier-free guidance for location-aware semantic guidance in video.
Optimization of latents in MotionClone is gradient-based, which occupies a certain amount of GPU memory.
We default to using higher resolution settings (now adjustable), which also contributes to GPU memory usage.

We will soon release a version that requires less GPU memory.