search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.37k stars 2.4k forks source link

[bug]: CUDA out of memory error with 3.1.1. No previous issue with 3.1.0. #4573

Open JakeSSRN opened 1 year ago

JakeSSRN commented 1 year ago

Is there an existing issue for this?

OS

Windows

GPU

cuda

VRAM

8 GB

What version did you experience this issue on?

3.1.1

What happened?

I'm getting the typical 'CUDA out of memory' error when generating with SDXL and SD2.1. With SD2.1 I even generate at 512x512 and 1 image batch size. In 3.1.0 I could generate a batch size of 3 at 1024x1024 with SDXL without running into problems. What gives?

System RAM cache is set to 12 of 16 GB and VRAM cache is set to 0 of 8 GB.

I noticed that the "free GPU memory after every generation" option is gone. I suspect this has something to do with it. Also, the whole browser crashes as soon as the error is thrown and I have to restart both the browser and Invoke to get it working again.

I've posted in the help section of the discord two days in a row now with absolutely zero response. I'd appreciate some insight, please.

Screenshots

Screenshot 2023-09-17 140202

Additional context

Probably unrelated: how I do I get Invoke to stop opening an new tab with the asset images in it every time I generate with the unified canvas?

Contact Details

No response

Millu commented 1 year ago

Sorry you message was missed on Discord - just responded there!

Petterson19 commented 1 year ago

I'm with the same problem and I can't join to discord

schoenid commented 1 year ago

Same to me ... except I'm on Ubuntu 22.04. Seems not to be a model problem. If I use the CPU only, it works, but it needs just 10+ times longer.

Petterson19 commented 1 year ago

Same to me ... except I'm on Ubuntu 22.04. Seems not to be a model problem. If I use the CPU only, it works, but it needs just 10+ times longer.

I found a solution, just downloaded CUDA from Nvidia and it's running smoothly now!

schoenid commented 1 year ago

I found a solution, just downloaded CUDA from Nvidia and it's running smoothly now!

@Petterson19 Thank you for the hint!

Unfortunately I've got it already installed.

~~>>> wrong <<< On SDXL: I can use CUDA up to a resolution of ~256x256. ~~

For higher resolutions: I have to switch to CPU only.

"nvidi-smi" from the terminal prints me the status of the GPU to the screen, with a list of all processes, which are using the GPU. So I can close any process, which is not really needed. It helps to increase the maximum resolution a little bit.

JakeSSRN commented 1 year ago

I found a solution, just downloaded CUDA from Nvidia and it's running smoothly now!

I uninstalled and reinstalled both CUDA toolkit and visual studio and restarted in between each step. No improvement.

Petterson19 commented 1 year ago

I found a solution, just downloaded CUDA from Nvidia and it's running smoothly now!

I uninstalled and reinstalled both CUDA toolkit and Visual Studio and restarted in between each step. No improvement.

Why didn't you use phyton in prompt or PowerShell? just too easy to install it

schoenid commented 1 year ago

So I made a mistake. The selected model I thougt was SDXL was SD2.1 only.

On SDXL and CUDA enabled I've got always Memory Errors:

[2023-09-21 00:36:29,950]::[InvokeAI]::ERROR --> Error while invoking:
CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 7.78 GiB total capacity; 5.80 GiB already allocated; 61.69 MiB free; 6.04 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Hm, the message means: 61.69 MiB are free, but 26.00 MiB can't be allocated. (Why ???) The PYTORCH_CUDA_ALLOC_CONF is a matter for the developer, as I see. I can't do this setting.

So SDXL is not usable on CUDA in invoke ai v3.1.1.

Remark nvidia-smi (CUDA-information):

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0  On |                  N/A |
|  0%   50C    P3    37W / 170W |   7879MiB /  8192MiB |      4%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1443      G   /usr/lib/xorg/Xorg                284MiB |
|    0   N/A  N/A      2268      G   /usr/bin/kwin_x11                  66MiB |
|    0   N/A  N/A      2316      G   /usr/bin/plasmashell               94MiB |
|    0   N/A  N/A      2785      G   ...402625461174669194,262144      166MiB |
|    0   N/A  N/A     55892      C   ...keAI/.venv/bin/python3.10     7260MiB |
+-----------------------------------------------------------------------------+

Processe:

JakeSSRN commented 1 year ago

The PYTORCH_CUDA_ALLOC_CONF is a matter for the developer, as I see. I can't do this setting. So SDXL is not usable on CUDA in invoke ai v3.1.1

I'm at a loss here. I guess I'll just have to wait for an update.

JakeSSRN commented 1 year ago

In a last ditch effort, I did a clean install of Windows 11. Invoke v3.1.1 and SDXL now work without issue. Still no idea what the problem was.

schoenid commented 1 year ago

I didn't realize, that the amount of used GPU memory (VRAM) can be reduced in setup: image

So easy! Now everything works fine!

icamps commented 2 months ago

Having this same error with InvokeAI 4.2.7.

No option in setup to reduce the amount of memory used, either option to "free GPU memory after every generation".

My box is running Ubuntu 22.04 and CUDA 12.4.