search

nixified-ai / flake

A Nix flake for many AI projects
GNU Affero General Public License v3.0
624 stars 69 forks source link

InvokeAI doesn't work (AMD) #83

Open mrnetlex opened 4 months ago

mrnetlex commented 4 months ago

I tried installing InvokeAI according to README with nix run github:nixified-ai/flake#invokeai-amd. During installation (after InvokeAI configuration screen) I got error:

2024-02-18 22:32:36,752]::[InvokeAI]::INFO --> Installing https://civitai.com/api/download/models/63006 [10/11]
Traceback (most recent call last):
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/bin/.invokeai-configure-wrapped", line 9, in <module>
    sys.exit(invokeai_configure())
             ^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/backend/install/invokeai_configure.py", line 946, in main
    process_and_execute(opt, models_to_download)
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/frontend/install/model_install.py", line 697, in process_and_execute
    installer.install(selections)
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/backend/install/model_install_backend.py", line 217, in install
    self.heuristic_import(path)
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/backend/install/model_install_backend.py", line 266, in heuristic_import
    models_installed.update({str(model_path_id_or_url): self._install_url(model_path_id_or_url)})
                                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/backend/install/model_install_backend.py", line 313, in _install_url
    location = download_with_resume(url, Path(staging))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/backend/util/util.py", line 290, in download_with_resume
    file_name = re.search('filename="(.+)"', resp.headers.get("Content-Disposition")).group(1)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6/lib/python3.11/re/__init__.py", line 176, in search
    return _compile(pattern, flags).search(string)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'

If I try to run nix run github:nixified-ai/flake#invokeai-amd once more, webui will launch but when trying to generate image it spits out error message in terminal:

Task exception was never retrieved
future: <Task finished name='Task-87' coro=<AsyncServer._handle_event_internal() done, defined at /nix/store/mv7p2fncq5qnlc1xsmzl60ppicmc1a7d-python3.11-python-socketio-5.9.0/lib/python3.11/site-packages/socketio/asyncio_server.py:513> exception=TypeError("object NoneType can't be used in 'await' expression")>
Traceback (most recent call last):
  File "/nix/store/mv7p2fncq5qnlc1xsmzl60ppicmc1a7d-python3.11-python-socketio-5.9.0/lib/python3.11/site-packages/socketio/asyncio_server.py", line 515, in _handle_event_internal
    r = await server._trigger_event(data[0], namespace, sid, *data[1:])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/mv7p2fncq5qnlc1xsmzl60ppicmc1a7d-python3.11-python-socketio-5.9.0/lib/python3.11/site-packages/socketio/asyncio_server.py", line 549, in _trigger_event
    ret = await handler(*args)
          ^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/app/api/sockets.py", line 33, in _handle_sub_queue
    await self.__sio.enter_room(sid, data["queue_id"])
TypeError: object NoneType can't be used in 'await' expression
[2024-02-18 22:33:35,212]::[uvicorn.access]::INFO --> 127.0.0.1:34536 - "GET /socket.io/?EIO=4&transport=polling&t=Os-8uHf&sid=2juFt2qYQJUb79BNAAAA HTTP/1.1" 200
[2024-02-18 22:33:43,941]::[uvicorn.access]::INFO --> 127.0.0.1:40234 - "POST /api/v1/queue/default/enqueue_batch HTTP/1.1" 200
[2024-02-18 22:33:43,948]::[uvicorn.access]::INFO --> 127.0.0.1:40234 - "GET /api/v1/queue/default/status HTTP/1.1" 200
[2024-02-18 22:33:43,975]::[uvicorn.access]::INFO --> 127.0.0.1:40234 - "GET /api/v1/queue/default/list HTTP/1.1" 200
[2024-02-18 22:33:44,093]::[InvokeAI]::INFO --> Loading model /home/netlex/invokeai/models/sd-1/main/stable-diffusion-v1-5, type sd-1:main:tokenizer
[2024-02-18 22:33:44,517]::[InvokeAI]::INFO --> Loading model /home/netlex/invokeai/models/sd-1/main/stable-diffusion-v1-5, type sd-1:main:text_encoder
[2024-02-18 22:33:47,313]::[InvokeAI]::ERROR --> Traceback (most recent call last):
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/app/services/processor.py", line 106, in __process
    outputs = invocation.invoke_internal(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/app/invocations/baseinvocation.py", line 610, in invoke_internal
    output = self.invoke(context)
             ^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/nv9rvrzc2nz1wsnkmsyghw81v0l26xb4-python3.11-torch-2.0.1/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/app/invocations/compel.py", line 124, in invoke
    c, options = compel.build_conditioning_tensor_for_conjunction(conjunction)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/yfxmbz2y141lcy22m1ryn9xfh9nb3sbl-python3.11-compel-2.0.2/lib/python3.11/site-packages/compel/compel.py", line 186, in build_conditioning_tensor_for_conjunction
    this_conditioning, this_options = self.build_conditioning_tensor_for_prompt_object(p)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/yfxmbz2y141lcy22m1ryn9xfh9nb3sbl-python3.11-compel-2.0.2/lib/python3.11/site-packages/compel/compel.py", line 218, in build_conditioning_tensor_for_prompt_object
    return self._get_conditioning_for_flattened_prompt(prompt), {}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/yfxmbz2y141lcy22m1ryn9xfh9nb3sbl-python3.11-compel-2.0.2/lib/python3.11/site-packages/compel/compel.py", line 282, in _get_conditioning_for_flattened_prompt
    return self.conditioning_provider.get_embeddings_for_weighted_prompt_fragments(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/yfxmbz2y141lcy22m1ryn9xfh9nb3sbl-python3.11-compel-2.0.2/lib/python3.11/site-packages/compel/embeddings_provider.py", line 120, in get_embeddings_for_weighted_prompt_fragments
    base_embedding = self.build_weighted_embedding_tensor(tokens, per_token_weights, mask, device=device)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/yfxmbz2y141lcy22m1ryn9xfh9nb3sbl-python3.11-compel-2.0.2/lib/python3.11/site-packages/compel/embeddings_provider.py", line 357, in build_weighted_embedding_tensor
    empty_z = self._encode_token_ids_to_embeddings(empty_token_ids)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/yfxmbz2y141lcy22m1ryn9xfh9nb3sbl-python3.11-compel-2.0.2/lib/python3.11/site-packages/compel/embeddings_provider.py", line 390, in _encode_token_ids_to_embeddings
    text_encoder_output = self.text_encoder(token_ids,
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/nv9rvrzc2nz1wsnkmsyghw81v0l26xb4-python3.11-torch-2.0.1/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/9qlsldh63laja5vb31685hqa5h2pf5kb-python3.11-transformers-4.35.2/lib/python3.11/site-packages/transformers/models/clip/modeling_clip.py", line 800, in forward
    return self.text_model(
           ^^^^^^^^^^^^^^^^
  File "/nix/store/nv9rvrzc2nz1wsnkmsyghw81v0l26xb4-python3.11-torch-2.0.1/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/9qlsldh63laja5vb31685hqa5h2pf5kb-python3.11-transformers-4.35.2/lib/python3.11/site-packages/transformers/models/clip/modeling_clip.py", line 693, in forward
    hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/nv9rvrzc2nz1wsnkmsyghw81v0l26xb4-python3.11-torch-2.0.1/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/9qlsldh63laja5vb31685hqa5h2pf5kb-python3.11-transformers-4.35.2/lib/python3.11/site-packages/transformers/models/clip/modeling_clip.py", line 216, in forward
    inputs_embeds = self.token_embedding(input_ids)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/nv9rvrzc2nz1wsnkmsyghw81v0l26xb4-python3.11-torch-2.0.1/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/nv9rvrzc2nz1wsnkmsyghw81v0l26xb4-python3.11-torch-2.0.1/lib/python3.11/site-packages/torch/nn/modules/sparse.py", line 162, in forward
    return F.embedding(
           ^^^^^^^^^^^^
  File "/nix/store/nv9rvrzc2nz1wsnkmsyghw81v0l26xb4-python3.11-torch-2.0.1/lib/python3.11/site-packages/torch/nn/functional.py", line 2210, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: HIP error: the operation cannot be performed in the present state
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing HIP_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.

[2024-02-18 22:33:47,320]::[InvokeAI]::ERROR --> Error while invoking:
HIP error: the operation cannot be performed in the present state
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing HIP_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.

Launching with HIP_LAUNCH_BLOCKING=1 doesn't change the error message.

System information:

MatthewCroughan commented 4 months ago

I do not own an AMD GPU, and do not have adequate funding to resolve that issue. So testing is not possible.

Are you running NixOS? If so, show me your configuration for your AMD driver.

mrnetlex commented 4 months ago

Yes, I'm running NixOS (unstable with use of flakes). Relevant part of configuration.nix:

  services.xserver.videoDrivers = [ "amdgpu" ];

  hardware.opengl.extraPackages = [ pkgs.amdvlk ];

I also use:

environment.variables = {
    HSA_OVERRIDE_GFX_VERSION = "10.3.0";
    AMD_VULKAN_ICD = "RADV";
  };

to use mesa by default instead of amdvlk and HSA_OVERRIDE_GFX_VERSION = "10.3.0"; used to help with running stable diffusion.

MatthewCroughan commented 4 months ago

HSA_OVERRIDE_GFX_VERSION is already set by the wrapper here https://github.com/nixified-ai/flake/blob/63339e4c8727578a0fe0f2c63865f60b6e800079/projects/invokeai/package.nix#L100, all I can say is that AMD GPU is difficult, and I am not sure I can help until I have the ability to test it on real hardware.

mrnetlex commented 4 months ago

It's understandable. Thanks for trying to help.

Blankjr commented 3 months ago

Related Problem:

nix run github:nixified-ai/flake#invokeai-amd
2024-03-17 21:27:29.275792898 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1827 CreateInferencePybindStateModule] Init provider bridge failed.
[2024-03-17 21:27:31,260]::[InvokeAI]::INFO --> Patchmatch initialized
/nix/store/knqd0zgkmj3pajqcmh785qc6m8hjf0hc-python3.11-torchvision-0.15.2/lib/python3.11/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(

An exception has occurred: /home/christian/invokeai/models/core/convert/CLIP-ViT-bigG-14-laion2B-39B-b160k is missing
== STARTUP ABORTED ==
** One or more necessary files is missing from your InvokeAI root directory **
** Please rerun the configuration script to fix this problem. **
** From the launcher, selection option [7]. **
** From the command line, activate the virtual environment and run "invokeai-configure --yes --skip-sd-weights" **
** (To skip this check completely, add "--ignore_missing_core_models" to your CLI args. Not installing these core models will prevent the loading of some or all .safetensors and .ckpt files. However, you can always come back and install these core models in the future.)
Press any key to continue..

System Information:

Fedora 
AMD RX 6700XT
32GB RAM

Nix Config (system wide installed):

build-users-group = nixbld
experimental-features = nix-command flakes repl-flake configurable-impure-env auto-allocate-uids
auto-optimise-store = true
bash-prompt-prefix = (nix:$name)\040
max-jobs = auto
extra-nix-path = nixpkgs=flake:nixpkgs
upgrade-nix-store-path-url = https://install.determinate.systems/nix-upgrade/stable/universal

# Define trusted substituters
trusted-substituters = https://ai.cachix.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= ai.cachix.org-1:N9dzRK+alWwoKXQlnn0H6aUx0lU/mspIoz8hMvGvbbc=
# Optionally, define other substituters if needed
substituters = https://cache.nixos.org https://ai.cachix.org
sc941737 commented 1 month ago

Related Problem:

nix run github:nixified-ai/flake#invokeai-amd
2024-03-17 21:27:29.275792898 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1827 CreateInferencePybindStateModule] Init provider bridge failed.
[2024-03-17 21:27:31,260]::[InvokeAI]::INFO --> Patchmatch initialized
/nix/store/knqd0zgkmj3pajqcmh785qc6m8hjf0hc-python3.11-torchvision-0.15.2/lib/python3.11/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(

An exception has occurred: /home/christian/invokeai/models/core/convert/CLIP-ViT-bigG-14-laion2B-39B-b160k is missing
== STARTUP ABORTED ==
** One or more necessary files is missing from your InvokeAI root directory **
** Please rerun the configuration script to fix this problem. **
** From the launcher, selection option [7]. **
** From the command line, activate the virtual environment and run "invokeai-configure --yes --skip-sd-weights" **
** (To skip this check completely, add "--ignore_missing_core_models" to your CLI args. Not installing these core models will prevent the loading of some or all .safetensors and .ckpt files. However, you can always come back and install these core models in the future.)
Press any key to continue..

I have this exact same error (the missing file is the same one as well). Running NixOS on AMD 7900XTX GPU, Ryzen 9 3900 CPU, 32GB RAM.

starlightsys commented 1 week ago

Related problem:

I have this same issue, except with an additional deprecation notice for torchvision.transforms.functional_tensor.

Here's the result of running it for the first time:

$ nix run github:nixified-ai/flake#invokeai-amd
State directory does not exist, running invokeai-configure
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.
0it [00:00, ?it/s]
2024-06-26 03:21:20.365665602 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1827 CreateInferencePybindStateModule] Init provider bridge failed.
[2024-06-26 03:21:21,065]::[InvokeAI]::INFO --> Initializing InvokeAI runtime directory
[2024-06-26 03:21:21,248]::[InvokeAI]::WARNING --> The file /home/username/invokeai/configs/models.yaml was not found. Initializing a new file
[2024-06-26 03:21:21,250]::[InvokeAI]::INFO --> Scanning /home/username/invokeai/models for new models
[2024-06-26 03:21:21,287]::[InvokeAI]::INFO --> Scanned 9 files and directories, imported 0 models
Traceback (most recent call last):
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/bin/.invokeai-configure-wrapped", line 9, in <module>
    sys.exit(invokeai_configure())
             ^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/backend/install/invokeai_configure.py", line 930, in main
    init_options, models_to_download = run_console_ui(opt, new_init_file)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/10b4kyq33crhfcpw3iiqvc4025bm6129-python3.11-InvokeAI-3.3.0post3/lib/python3.11/site-packages/invokeai/backend/install/invokeai_configure.py", line 749, in run_console_ui
    torch.multiprocessing.set_start_method("spawn")
  File "/nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6/lib/python3.11/multiprocessing/context.py", line 247, in set_start_method
    raise RuntimeError('context has already been set')
RuntimeError: context has already been set

And the second time:

$ nix run github:nixified-ai/flake#invokeai-amd
2024-06-26 03:24:56.048227459 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1827 CreateInferencePybindStateModule] Init provider bridge failed.
/nix/store/knqd0zgkmj3pajqcmh785qc6m8hjf0hc-python3.11-torchvision-0.15.2/lib/python3.11/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(
[2024-06-26 03:25:01,095]::[InvokeAI]::INFO --> Patchmatch initialized

An exception has occurred: /home/username/invokeai/models/core/convert/CLIP-ViT-bigG-14-laion2B-39B-b160k is missing
== STARTUP ABORTED ==
** One or more necessary files is missing from your InvokeAI root directory **
** Please rerun the configuration script to fix this problem. **
** From the launcher, selection option [7]. **
** From the command line, activate the virtual environment and run "invokeai-configure --yes --skip-sd-weights" **
** (To skip this check completely, add "--ignore_missing_core_models" to your CLI args. Not installing these core models will prevent the loading of some or all .safetensors and .ckpt files. However, you can always come back and install these core models in the future.)
Press any key to continue...

No selection menu ever appears, and I don't really know what more to do about that.

System information

OS: NixOS 24.11 (Vicuna) x86_64 Kernel: Linux 6.6.35 CPU: AMD Ryzen 7 5700G (16) @ 3.80 GHz GPU: AMD Radeon RX 6800