search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.34k stars 2.4k forks source link

[question]: switch to using CPU as the GPU is too low-spec #2686

Closed einarpetersen closed 1 year ago

einarpetersen commented 1 year ago

Is there an existing issue for this?

OS

Linux

GPU

cuda

VRAM

nvidia-smi --query-gpu=memory.total --format=csv,noheader 1999 MiB

What happened?

So I recently got my Nvidia up and running in Debian 11, however now because InvokeAI is trying to use CUDA I'm getting the following error:

File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 987, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 74.00 MiB (GPU 0; 1.95 GiB total capacity; 1.21 GiB already allocated; 21.38 MiB free; 1.26 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Here is an entire dump from starting --web

A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton'

** trying to reload previous model

Loading diffusers model from runwayml/stable-diffusion-v1-5 | Using faster float16 precision | Loading diffusers VAE from stabilityai/sd-vae-ft-mse Fetching 15 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 19636.25it/s] An error occurred while attempting to initialize the model: "CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 1.95 GiB total capacity; 1.99 MiB already allocated; 1.12 MiB free; 2.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF" This can be caused by a missing or corrupted models file, and can sometimes be fixed by (re)installing the models.

** This can be caused by a missing or corrupted models file, and can sometimes be fixed by (re)installing the models. Do you want to run invokeai-configure script to select and/or reinstall models? [y] n

Opening the Web UI - I see GREEN Connected Dreamlike-diffusion-1.0

Trying to generate produces the following error

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 1.95 GiB total capacity; 24.36 MiB already allocated; 512.00 KiB free; 26.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I can't seem to load any models, ger ERROR in the UI when I do - I know the models are installed, have downloaded multiple times and it was working prior in 2.3.0+a0 - Now after "upgrade" new UI with negative prompt says 2.3.0 weird

Has anyone gotten further - I can see a ton of people asking about very similar issues

Operating System: Debian GNU/Linux 11 KDE Plasma Version: 5.20.5 KDE Frameworks Version: 5.78.0 Qt Version: 5.15.2 Kernel Version: 5.10.0-21-amd64 OS Type: 64-bit Processors: 8 × Intel® Core™ i7-4790K CPU @ 4.00GHz Memory: 15.6 GiB of RAM Graphics Processor: NVIDIA GeForce GTX 750 Ti/PCIe/SSE2

System free -h total used free shared buff/cache available Mem: 15Gi 2.1Gi 9.8Gi 39Mi 3.7Gi 13Gi Swap: 975Mi 20Mi 955Mi

GPU:

nvidia-smi --query-gpu=memory.total --format=csv,noheader 1999 MiB

Thu Feb 16 18:10:10 2023
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.103.01 Driver Version: 470.103.01 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... On | 00000000:01:00.0 On | N/A | | 22% 30C P0 2W / 38W | 343MiB / 1999MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 843 G /usr/lib/xorg/Xorg 129MiB | | 0 N/A N/A 1392 G /usr/bin/kwin_x11 29MiB | | 0 N/A N/A 1460 G /usr/bin/plasmashell 31MiB | | 0 N/A N/A 1564 G ...b/firefox-esr/firefox-esr 119MiB | | 0 N/A N/A 3189 G /usr/bin/systemsettings5 25MiB | +-----------------------------------------------------------------------------+

Screenshots

No response

Additional context

No response

Contact Details

No response

einarpetersen commented 1 year ago

A user said: According to that, your GPU falls under the requirements for InvokeAI - check them out here: https://invoke-ai.github.io/InvokeAI/#hardware-requirements

— Today at 6:37 PM Works perfectly just using the CPU - Why would adding GPU make it fail - But yeah I see the recommendations.... hmmm Or I should perhaps say used to work 🙂

— Today at 6:48 PM I will see if I can downgrade to CPU again and if it will work in the new UI Einar Petersen (Comics Creator)

— Today at 7:22 PM did not help to do the : pip install InvokeAI --use-pep517 --extra-index-url https://download.pytorch.org/whl/cpu install.... still getting File "/home/ep/InvokeAI/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 987, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 74.00 MiB (GPU 0; 1.95 GiB total capacity; 1.24 GiB already allocated; 6.06 MiB free; 1.28 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

einarpetersen commented 1 year ago

So somehow InvokeAI still tries to use the CUDA device despite it being installed as CPU only...

ebr commented 1 year ago
  1. An easy fix might be export CUDA_VISIBLE_DEVICES=-1 (set in your shell environment as persistently as you like)
  2. you could try to pip uninstall torch and then pip install torch --force-reinstall --extra-index-url https://download.pytorch.org/whl/cpu. This should ensure you are using the CPU version of torch.
  3. A "nuclear" option might be to remove your InvokeAI virtual environment completely, purge all the pip caches, and try to reinstall again to make sure you're not getting a GPU torch from somewhere.

But if (1) works, that might be all that's needed.

einarpetersen commented 1 year ago

Ended up doing a nuclear - I can generate images again, however masking/in-painting no longer works....

einarpetersen commented 1 year ago

As in no mask appears no matter brush color picked layer chosen - Simply non existent - eraser has also disappeared

ebr commented 1 year ago

no mask appears no matter brush color picked layer chosen

This seems to be a UI issue; couldn't reproduce it on my end. Could you please try the Reset Web UI button under Settings and see if it helps?

github-actions[bot] commented 1 year ago

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

einarpetersen commented 1 year ago

Did a pip upgrade and then inpaint etc. started working again

github-actions[bot] commented 1 year ago

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.