search

horseee / DeepCache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free
https://horseee.github.io/Diffusion_DeepCache/
Apache License 2.0
687 stars 32 forks source link

How much GPU memory is needed to complete the SVD experiment? #19

Closed Peyton-Chen closed 6 months ago

Peyton-Chen commented 6 months ago

In my SVD experiment, OOM occurred on the 80G A100. So I would like to ask how much GPU memory can support SVD experiments. Thanks!

horseee commented 6 months ago

Hi @Peyton-Chen ,

The inference of SVD requires <13G memory. You can try this command:

python main.py --model_type svd
Peyton-Chen commented 6 months ago

Thank you for your response！ I've successfully resolved the issue. It appears that the root cause was related to my package version.

horseee commented 6 months ago

One possible cause for this issue:

If the error RuntimeError: expected scalar type float but found c10::Half occurs when running SVD with fp16, or the out-of-memory problem occurs after switching to fp32, please refer to the answer under this issue and increase the version of torch to 2.X