search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.34k stars 2.4k forks source link

[bug]: invokeai-rocm container doesn't support gpus #7006

Open nmcbride opened 1 week ago

nmcbride commented 1 week ago

Is there an existing issue for this problem?

Operating system

Linux

GPU vendor

AMD (ROCm)

GPU model

RX 7900 XTX, RX 7700S

GPU VRAM

26GB, 8GB

Version number

invokeai-rocm

Browser

firefox

Python dependencies

No response

What happened

I am trying to use the container version like this:

--device /dev/kfd --device /dev/dri --volume ./:/invokeai -p 9090:9090 --name invokeai ghcr.io/invoke-ai/invokeai:main-rocm

However it doesn't seem to detect either of my amd GPUs and falls back to CPU. It also says bitsandbytes doesn't have GPU support.

The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
[2024-10-02 05:54:19,891]::[InvokeAI]::INFO --> Patchmatch initialized
[2024-10-02 05:54:20,552]::[InvokeAI]::INFO --> Using torch device: CPU

ollama works fine with rocm I am not sure why this doesn't or how I can get it working?

What you expected to happen

I expect the container to start utilizing rocm and detecting the gpu

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

SadmL commented 1 day ago

Bare-metal affected too Using Installer with rocm option - RX 6700 XT GPU Works fine on < 5.0.2, but starting > 5.1 getting error The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable. Clearing pip cache isn't helping

Made systemd service

[Unit]
Description=InvokeAI

[Service]
ExecStart=/home/user/.local/invokeai/.venv/bin/invokeai-web
Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"
Environment="PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.95,max_split_size_mb:512"
Environment="INVOKEAI_ROOT=/home/user/.local/invokeai"
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=default.target