torch.cuda.OutOfMemoryError with SDXL

[Marcel0504]

Hello,

I've set up my notebook on Paperspace as per the instructions in TheLastBen/PPS, aiming to run StableDiffusion XL on a P4000 GPU. However, when attempting to generate an image, I encounter a CUDA out of memory error:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 10.00 MiB (GPU 0; 7.92 GiB total capacity; 6.79 GiB already allocated; 5.69 MiB free; 7.04 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I've followed all setup instructions to the letter and haven't deviated from the recommended settings. Despite the detailed error message, I'm unsure how to proceed to resolve this. Has anyone encountered a similar issue or has suggestions what I should do?

I previously tried to:

• delete pytorch and update to a newer version that supports the CUDA Version running on the GPU (12.0)
• use a higher PyTorch version
• use SD 1.5 instead of SDXL, which seems to work

Thank you very much!

[TheLastBen]

try again now

[Marcel0504]

Thanks for the quick response and for trying to resolve the issue, but unfortunately it still doesn't work right.

Now I don't get an error message, but when I open the WebUI and try to generate an image it shows "In queue 1/1" for a moment and then just loads forever but without generating any image. If I try to restart the Web UI it again goes into "Reloading..." state and then nothing happens.

I hope this helps. Thanks again for the support!

[autonomous1]

Am receiving the same CUDA out of memory error on a RunPod with template RunPod Fast Stable Diffusion runpod/stable-diffusion:fast-stable-diffusion-2.4.0, running on RTXA4500. It was working fine up until May 29, 2024

OutOfMemoryError: CUDA out of memory. Tried to allocate 9.41 GiB (GPU 0; 19.71 GiB total capacity; 16.71 GiB already allocated; 2.32 GiB free; 17.13 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF A: 16.81 GB, R: 18.59 GB, Sys: 18.8/19.7061 GB (95.6%)

[autonomous1]

The out-of-memory error was resolved by reducing the "resize to" (img2img) image parameters to a smaller image size. None of the suggested methods of changing the max split size were effective, including: os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "max_split_size_mb:"