[bug]: InvokeAI won't use GPU (CUDA/NVIDIA). Defaults to CPU

[FalxGod]

Is there an existing issue for this?

• [X] I have searched the existing issues

OS

Windows

GPU

cuda

VRAM

24GB

What happened?

Starting invoke.bat is always saying that it uses the CPU instead of my GPU (And it indeed tries to generate with the CPU). Here's the cmd output with the relevant line indicated by an arrow. The line should read ">> Using device_type cuda" not "CPU"

1. command-line
2. browser-based UI
3. open the developer console
Please enter 1, 2 or 3: 2
Starting the InvokeAI browser-based UI..
* Initializing, be patient...
>> Initialization file C:\Users\Falxie II/.invokeai found. Loading...
>> InvokeAI runtime directory is "J:\AI\Art\StableDiffusion\invokeAI"
>> Patchmatch initialized
>> GFPGAN Initialized
>> CodeFormer Initialized
>> ESRGAN Initialized
>> Using device_type CPU     <-----------------------------
>> Current VRAM usage:  0.00G
>> Scanning Model: stable-diffusion-1.5
>> Model Scanned. OK!!
>> Loading stable-diffusion-1.5 from J:\AI\Art\StableDiffusion\invokeAI\models\ldm\stable-diffusion-v1\v1-5-pruned-emaonly.ckpt
   | LatentDiffusion: Running in eps-prediction mode
   | DiffusionWrapper has 859.52 M params.
   | Making attention of type 'vanilla' with 512 in_channels
   | Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
   | Making attention of type 'vanilla' with 512 in_channels
   | Using more accurate float32 precision
   | Loading VAE weights from: J:\AI\Art\StableDiffusion\invokeAI\models\ldm\stable-diffusion-v1\vae-ft-mse-840000-ema-pruned.ckpt
>> Model loaded in 7.44s
>> Current embedding manager terms: *
>> Setting Sampler to k_euler_a

* --web was specified, starting web server...
>> Initialization file C:\Users\Falxie II/.invokeai found. Loading...
>> Started Invoke AI Web Server!
>> Default host address now 127.0.0.1 (localhost). Use --host 0.0.0.0 to bind any address.
>> Point your browser at http://127.0.0.1:9090

Screenshots

Here's a screenshot of a prompt being executed. Notice the time to completion in the bottom right corner, And this is my GPU during the generation process. Notice the utilization which also indicates that the process uses the CPU.

Additional context

I reinstalled InvokeAI several times and it's always the same. Automatic1111 WebUI uses the GPU just fine.

Contact Details

No response

[vis223]

When I added these 3 paths to my environment variables the source installer finally started working.

%SystemRoot%\system32 %SystemRoot% %SystemRoot%\System32\Wbem

also, make sure to check the paths in the small config file called .invokeai in the "user/'my_name'" folder. It seems to override your choices unless you edit the file. 1

[FalxGod]

When I added these 3 paths to my environment variables the source installer finally started working.

%SystemRoot%\system32 %SystemRoot% %SystemRoot%\System32\Wbem

also, make sure to check the paths in the small config file called .invokeai in the "user/'my_name'" folder. It seems to override your choices unless you edit the file. 1

I think you posted in the wrong thread? The installer works fine for me. And my .invokeai file doesn't cause any problems.

[cybromancy]

@FalxGod Yes known issue with using CPU instead of GPU. Discord link with ideas of doing the manual install, and updating pytorch. https://discord.com/channels/1020123559063990373/1047568895168483348 InvokeAI 2.0.2 was using GPU and then switched to 2.0 and now uses CPU. So whatever changed in the version or if in that time pytorch made 11.6 the latest version? As of this post now known fix. Thread with more info on how to check/ install pytorch version: https://github.com/invoke-ai/InvokeAI/discussions/1628#discussioncomment-4277763

[cybromancy]

@FalxGod pytorch doesnt work with python 3.11 and python doesn't like being deleted. Have to manually ctr+f 'python' in reg edit. Shows one entry delete then ctr+f again to find next.

here's from a clean install of python 3.10.8 pip install numpy

pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116

pip install virtualenv

virtualenv invokeai

cd c:/ to invokeAI folder (easiest way is to find invokeAI folder and right click the top address and copy) in cmd type cd ctr+v

copy environments-and-requirements\requirements-win-colab-cuda.txt requirements.txt

pip install --prefer-binary -r requirements.txt

python scripts/invoke.py --web

[FalxGod]

@cybromancy Ty I will look into it! Imma leave the issue open for now.

[Kallamamran]

Still no luck here then :/ Damn.... I think this should be one of the main issues to look at. How many want to sun sirapy slow on a CPU when they can gp lightning fast on a GPU?!

[Xerophayze]

I am having the same issue. RTX 3080 Ti. Its using CPU instead of GPU. This is from a fresh automated install.

[DivinAria]

Yeah I’m not going to even bother trying at this point.

[DivinAria]

Still no luck here then :/ Damn.... I think this should be one of the main issues to look at. How many want to sun sirapy slow on a CPU when they can gp lightning fast on a GPU?!

It is 100% the main issue. I wish I’d never come across this app tbh it’s been nothing but a tease and a headache.

[FalxGod]

EASY FIX:

1. Open enviroment.yml
2. change line 9,10,11 to

   • torchvision=0.13.1+cu116
     • torchaudio=0.12.1+cu116
     • pytorch=1.12.1+cu116

3. run install.bat
4. run invoke.bat Running cuda now

[Xerophayze]

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

1. Open enviroment.yml

2. change line 9,10,11 to

   • torchvision=0.13.1+cu116
   • torchaudio=0.12.1+cu116
   • pytorch=1.12.1+cu116

3. run install.bat

4. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

[Xerophayze]

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

1. Open enviroment.yml

2. change line 9,10,11 to

   • torchvision=0.13.1+cu116
   • torchaudio=0.12.1+cu116
   • pytorch=1.12.1+cu116

3. run install.bat

4. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

[Xerophayze]

so when I run the install.bat after making the changes to the yml file, it redownloads the file, removing the changes i just made.

On Mon, Dec 5, 2022 at 2:32 PM Echo T @.***> wrote:

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

1. Open enviroment.yml

2. change line 9,10,11 to

   • torchvision=0.13.1+cu116
   • torchaudio=0.12.1+cu116
   • pytorch=1.12.1+cu116

3. run install.bat

4. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

[Xerophayze]

I changed the file to read only so it could not replace the file, but this is what it shows. [image: image.png]

On Mon, Dec 5, 2022 at 2:38 PM Echo T @.***> wrote:

so when I run the install.bat after making the changes to the yml file, it redownloads the file, removing the changes i just made.

On Mon, Dec 5, 2022 at 2:32 PM Echo T @.***> wrote:

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

1. Open enviroment.yml

2. change line 9,10,11 to

   • torchvision=0.13.1+cu116
   • torchaudio=0.12.1+cu116
   • pytorch=1.12.1+cu116

3. run install.bat

4. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

[Xerophayze]

another thing, in the file its not line 9,10,11. its 10,11,12.

[image: image.png]

On Mon, Dec 5, 2022 at 2:40 PM Echo T @.***> wrote:

I changed the file to read only so it could not replace the file, but this is what it shows. [image: image.png]

On Mon, Dec 5, 2022 at 2:38 PM Echo T @.***> wrote:

so when I run the install.bat after making the changes to the yml file, it redownloads the file, removing the changes i just made.

On Mon, Dec 5, 2022 at 2:32 PM Echo T @.***> wrote:

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

1. Open enviroment.yml

2. change line 9,10,11 to

   • torchvision=0.13.1+cu116
   • torchaudio=0.12.1+cu116
   • pytorch=1.12.1+cu116

3. run install.bat

4. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

[FalxGod]

Here's what I have done in total before it worked for me (excerpt from discord): I also deleted the C.\Users[username].invokeai file before running install.bat and invoke.bat (after editing the lines; which I just realized by reading your comment and another in the github issue did nothing because it gets overwritten by running install.bat)

Im doubtful it will help, because the .invoke file says nothing about cuda...

I also did some steps from the manual install but not all. I ran:

1. git clone https://github.com/invoke-ai/InvokeAI.git
2. conda env update
3. conda activate invokeai
4. python scripts/invoke.py

Then I deleted the whole folder again. (Because I got lazy~~) After that I ran these 2 commands in my cmd:

1. pip install numpy
2. pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116

After that i edited the 3 lines in the enviroment.yml and it worked

[Kallamamran]

Yeah that didn't work (W10)

[Kallamamran]

2. pip install torch==1.12.1+cu116

That didn't work either

[FalxGod]

@Kallamamran what version of Python do you have installed? edit: If you have Python 11 then uninstall it and get python 10.x. PyTorch is not supported in version 11 of Python at the moment. (Had the same error when using Python 11)

[nazeerPro]

I Have installed invokeai 2.5.5 Completely Without error Now when i start invokeai.bat This appears

1. command-line
2. browser-based UI
3. open the developer console Please enter 1, 2 or 3 Now When i type 1 or 2 it won't work only typing 3 is working how to fix this

[VimCommando]

When you are looking at the Windows performance monitor, make sure you've expanded the details and changed one of the graphs to Cuda. The default load % shown could be for the 3D pipeline, not Cuda.

So when you're looking at if a patch worked or not, you want to see the Cuda load percentage, like in this example from my Windows 10 machine:

[mauwii]

@VimCommando When you start InvokeAI, Is one of the first lines >> Patchmatch initialized?

I am just currious, since I have the feeling that the precompiled dll for windows is only compiled with CPU Features. But this should of course not stop the other Modules from accessing the GPU (which is why it would be great if you have Patchmatch initialized 😅)

EDIT:

Just saw that we already have prove that patchmatch does not block the use of cuda:

[VimCommando]

Plot twist: my above statement is only true if hardware accelerated GPU scheduling is not enabled: https://devblogs.microsoft.com/directx/hardware-accelerated-gpu-scheduling/

Once you turn this on, then CUDA is no longer a separate category for the performance monitor, and the load reflects in the main utilization graph as expected.

I discovered this after upgrading my graphics card and trying to get a completely unrelated program running.

[rafalou38]

I fixed it with:

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html

(while in the environment)

You can check if cuda is working with:

$ python
Python 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

After that, I directly ran invoke.py.

[veetance]

I fixed it with:

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html

(while in the environment)

You can check if cuda is working with:

$ python
Python 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

After that, I directly ran invoke.py.

Yup this worked for me!!. i used invoke.bat

[SeanBannister]

The above fix didn't work for me but this similar one did: https://github.com/invoke-ai/InvokeAI/issues/2460#issuecomment-1416056021