search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.34k stars 2.4k forks source link

Critical Performance Drop in InvokeAI 4.2.x: Image Generation Slows to a Crawl #6364

Open zero2999 opened 4 months ago

zero2999 commented 4 months ago

Is there an existing issue for this problem?

Operating system

Linux

GPU vendor

Nvidia (CUDA)

GPU model

NVIDIA GeForce RTX 3060 Mobile / Max-Q

GPU VRAM

6GB

Version number

4.2.1

Browser

microsoft edga

Python dependencies

No response

What happened

Hello, I have been a user of InvokeAI since version 3.x, and up until version 4.1.x, it generated images efficiently in both S.D 1.5 and S.D XL. However, after upgrading to version 4.2.x, the performance has deteriorated significantly, with image generation times ranging from 15 to 30 minutes. I reviewed the documentation for potential solutions, but the current recommendation is to perform a clean uninstall and reinstall. I have executed this procedure four times, but the issue persists.

Assuming the problem might be related to a system update, I performed a complete OS reinstallation. Unfortunately, the performance issues remain unchanged, with the application still running sluggishly. I would appreciate any technical assistance or solutions to optimize and expedite the image generation process.

image

What you expected to happen

I expected image generation in InvokeAI to be fast and efficient, similar to previous versions (up to version 4.1.x), in both S.D 1.5 and S.D XL.

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

psychedelicious commented 4 months ago

Let's check a couple things:

zero2999 commented 4 months ago

patchmatch.patch_match: INFO - Compiling and loading c extensions from "/home/ppbg/invokeai/.venv/lib/python3.10/site-packages/patchmatch". patchmatch.patch_match: ERROR - patchmatch failed to load or compile (Command 'make clean && make' returned non-zero exit status 2.). patchmatch.patch_match: INFO - Refer to https://invoke-ai.github.io/InvokeAI/installation/060_INSTALL_PATCHMATCH/ for installation instructions. [2024-05-14 23:56:58,900]::[InvokeAI]::INFO --> Patchmatch not loaded (nonfatal) [2024-05-14 23:57:01,258]::[InvokeAI]::INFO --> Using torch device: CPU [2024-05-14 23:57:01,431]::[InvokeAI]::INFO --> cuDNN version: 8902 [2024-05-14 23:57:01,451]::[uvicorn.error]::INFO --> Started server process [51400] [2024-05-14 23:57:01,451]::[uvicorn.error]::INFO --> Waiting for application startup. [2024-05-14 23:57:01,451]::[InvokeAI]::INFO --> InvokeAI version 4.2.1 [2024-05-14 23:57:01,452]::[InvokeAI]::INFO --> Root directory = /home/ppbg/invokeai [2024-05-14 23:57:01,452]::[InvokeAI]::INFO --> Initializing database at /home/ppbg/invokeai/databases/invokeai.db [2024-05-14 23:57:01,462]::[InvokeAI]::INFO --> Pruned 3 finished queue items [2024-05-14 23:57:01,477]::[InvokeAI]::INFO --> Cleaned database (freed 0.02MB) [2024-05-14 23:57:01,478]::[uvicorn.error]::INFO --> Application startup complete. [2024-05-14 23:57:01,478]::[uvicorn.error]::INFO --> Uvicorn running on http://127.0.0.1:9090 (Press CTRL+C to quit)

And that's even though I chose option 1 for Nvidia during installation. As I mentioned earlier, it used to generate images very quickly, but now it's incredibly slow when generating an image

zero2999 commented 4 months ago

invokeai.yaml

Internal metadata - do not edit:

schema_version: 4.0.1

Put user settings here - see https://invoke-ai.github.io/InvokeAI/features/CONFIGURATION/:

psychedelicious commented 4 months ago

It seems torch didn't install the correct version of itself or is falling back to CPU. What do you get when you run nvidia-smi in a terminal?

zero2999 commented 4 months ago

ppbg@ppbg-IdeaPad-Gaming-3-15ACH6:~/.cache$ nvidia-smi Command 'nvidia-smi' not found, but can be installed with: sudo apt install nvidia-utils-390 # version 390.157-0ubuntu0.22.04.2, or sudo apt install nvidia-utils-418-server # version 418.226.00-0ubuntu5~0.22.04.1 sudo apt install nvidia-utils-450-server # version 450.248.02-0ubuntu0.22.04.1 sudo apt install nvidia-utils-470 # version 470.223.02-0ubuntu0.22.04.1 sudo apt install nvidia-utils-470-server # version 470.223.02-0ubuntu0.22.04.1 sudo apt install nvidia-utils-525 # version 525.147.05-0ubuntu0.22.04.1 sudo apt install nvidia-utils-525-server # version 525.147.05-0ubuntu0.22.04.1 sudo apt install nvidia-utils-535 # version 535.129.03-0ubuntu0.22.04.1 sudo apt install nvidia-utils-535-server # version 535.129.03-0ubuntu0.22.04.1 sudo apt install nvidia-utils-510 # version 510.60.02-0ubuntu1 sudo apt install nvidia-utils-510-server # version 510.47.03-0ubuntu3

psychedelicious commented 4 months ago

I'm pretty sure you get nvidia-smi when you install your GPU drivers. Missing or nonfunctional drives could result in the app falling back to CPU, causing generation to be very slow.

Please ensure your GPU drivers are installed - reinstall if they are not - then run nvidia-smi so we can confirm that your drivers are installed and functioning.