search

huggingface / candle

Minimalist ML framework for Rust
Apache License 2.0
15.77k stars 946 forks source link

Stable Diffusion 3.5 Large CUDA OUT_OF_MEMORY on RTX 3090 #2597

Open danielclough opened 21 hours ago

danielclough commented 21 hours ago

When I run cargo run --example stable-diffusion-3 --release --features=cuda -- --which 3.5-large --prompt "pretty picture" I am get Error: DriverError(CUDA_ERROR_OUT_OF_MEMORY, "out of memory") with Stable Diffusion 3.5 Large and Turbo.

According to this chart from stability.ai they should run on an RTX 3090.

chart

LaurentMazare commented 21 hours ago

That seems odd, we made a couple optimizations to memory usage following #2574 and in the end, SD 3.5 large was reported to work well on a GPU with only 20GB of memory. Maybe there are some other processes using the memory? If not it would be good to run a nsys profile to see when the memory is being used.

danielclough commented 20 hours ago

There are no other processes running.

How would you like me to run nsys?

Here's some system info:


cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.1 LTS"

rustc --version
rustc 1.81.0 (eeb90cda1 2024-09-04)

cargo --version
cargo 1.81.0 (2dbb1af80 2024-08-20)

NVIDIA-SMI 560.35.03              Driver Version: 560.35.03      CUDA Version: 12.6
...