search

lllyasviel / Fooocus

Focus on prompting and generating
GNU General Public License v3.0
40.74k stars 5.69k forks source link

AssertionError: Torch not compiled with CUDA enabled #113

Closed arbidroid closed 9 months ago

arbidroid commented 1 year ago

I tried to install manually on Windows but keep getting the following during startup

Traceback (most recent call last): File "C:\dev\miniconda3\envs\fooocus\lib\threading.py", line 1016, in _bootstrap_inner self.run() File "C:\dev\miniconda3\envs\fooocus\lib\threading.py", line 953, in run self._target(*self._args, *self._kwargs) File "c:\dev\git\Fooocus\modules\async_worker.py", line 15, in worker import modules.default_pipeline as pipeline File "c:\dev\git\Fooocus\modules\default_pipeline.py", line 1, in import modules.core as core File "c:\dev\git\Fooocus\modules\core.py", line 7, in import comfy.model_management File "C:\dev\git\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\model_management.py", line 104, in total_vram = get_total_memory(get_torch_device()) / (1024 1024) File "C:\dev\git\Fooocus\repositories\ComfyUI-from-StabilityAI-Official\comfy\model_management.py", line 74, in get_torch_device return torch.device(torch.cuda.current_device()) File "C:\dev\miniconda3\envs\fooocus\lib\site-packages\torch\cuda__init__.py", line 674, in current_device _lazy_init() File "C:\dev\miniconda3\envs\fooocus\lib\site-packages\torch\cuda__init__.py", line 239, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

I have CUDA toolkit installed. Tried Automatic1111 and it works fine.

markkrueg commented 1 year ago

I am getting similar errors when I try to generate images on my manual (using pip) Ubuntu Linux Fooocus installation: xFormers wasn't build with CUDA support

If I do 

pip show torch
Name: torch
Version: 2.0.1+cu118
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3
Location: /home/aitest/.local/lib/python3.10/site-packages
Requires: filelock, jinja2, networkx, sympy, triton, typing-extensions
Required-by: accelerate, basicsr, clean-fid, clip, facexlib, gfpgan, kornia, lpips, open-clip-torch, pytorch-lightning, realesrgan, timm, torchaudio, torchdiffeq, torchmetrics, torchsde, torchvision, triton, xformers

Not sure what I need to do to get CUDA support working.

markkrueg commented 1 year ago

Aha; I also see this when I launch Fooocus:

WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 1.13.1+cu117 with CUDA 1107 (you have 2.0.1+cu118)
    Python  3.10.9 (you have 3.10.6)
  Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details

I will try to re-install xformers per URL.

lllyasviel commented 1 year ago

Hi our linux tests have passed on Google Colab and many main stream environments like runpod, etc. We are willing to take a look at the problem, and also we can take a look at that Colab to see if there is any problem between the standard linux in Colab and your local linux configurations.

arbidroid commented 1 year ago

@markkrueg in your case torch has cuda support but compiled for different version. I have no cuda support.

import torch print("version", torch.version) version 2.0.1+cpu print("version", torch.cuda.is_available()) version False

Should I be able to run this manually on windows?

pcartagena commented 1 year ago

I tried to install manually on Windows but keep getting the following during startup . . . AssertionError: Torch not compiled with CUDA enabled

I have CUDA toolkit installed. Tried Automatic1111 and it works fine.

I was able to resolve same issue by forcibly reinstalling just the torch files via pytorch's instructions (https://pytorch.org/get-started/locally/) -

python -m pip install --upgrade --force-reinstall --no-cache-dir --no-deps torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

cufarvid commented 1 year ago

I tried to install manually on Windows but keep getting the following during startup . . . AssertionError: Torch not compiled with CUDA enabled I have CUDA toolkit installed. Tried Automatic1111 and it works fine.

I was able to resolve same issue by forcibly reinstalling just the torch files via pytorch's instructions (https://pytorch.org/get-started/locally/) -

python -m pip install --upgrade --force-reinstall --no-cache-dir --no-deps torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

Works on Windows. I have used https://download.pytorch.org/whl/cu118 as --index-url.

MidniteMekanix commented 1 year ago

What about if you don't have a cuda supported system and want to change it to CPU? What do I have to change?

WhiteBowlerHat commented 10 months ago

I am using : GPU : AMD Radeon RX 6600 (VRAM 8176 MB) CPU: AMD Ryzen 5 5600 6-Core Processor and I keep getting this error

C:\Users\theim\Projets\AI>.\python_embeded\python.exe -s Fooocus\entry_with_update.py --directml --lowvram
Already up-to-date
Update succeeded.
[System ARGV] ['Fooocus\\entry_with_update.py', '--directml', '--lowvram']
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.1.824
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Using directml with device:
Total VRAM 1024 MB, total RAM 16310 MB
Set vram state to: LOW_VRAM
Disabling smart memory management
Device: privateuseone
VAE dtype: torch.float32
Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --use-split-cross-attention
Refiner unloaded.
model_type EPS
adm 2816
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
extra keys {'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: C:\Users\theim\Projets\AI\Fooocus\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [C:\Users\theim\Projets\AI\Fooocus\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors].
Loaded LoRA [C:\Users\theim\Projets\AI\Fooocus\models\loras\sd_xl_offset_example-lora_1.0.safetensors] for UNet [C:\Users\theim\Projets\AI\Fooocus\models\checkpoints\juggernautXL_version6Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cpu, use_fp16 = False.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 6626011583025000839
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] a ship, sharp focus, intricate, detailed, rich deep colors, cinematic light, sleek, classic, fine detail, inspired, highly coherent, symmetry, vibrant, saturated, color, pristine, beautiful, epic, brilliant, great, awesome, magnificent, fabulous, romantic, elegant, luxury, very inspirational, artistic, peaceful, extremely lucid, iconic, ambient, fancy
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding negative #1 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (768, 1280)
Preparation time: 9.26 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
loading in lowvram mode 256.0
Traceback (most recent call last):
  File "C:\Users\theim\Projets\AI\Fooocus\modules\async_worker.py", line 803, in worker
    handler(task)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\theim\Projets\AI\Fooocus\modules\async_worker.py", line 735, in handler
    imgs = pipeline.process_diffusion(
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\theim\Projets\AI\Fooocus\modules\default_pipeline.py", line 361, in process_diffusion
    sampled_latent = core.ksampler(
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\theim\Projets\AI\Fooocus\modules\core.py", line 315, in ksampler
    samples = fcbh.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
  File "C:\Users\theim\Projets\AI\Fooocus\backend\headless\fcbh\sample.py", line 93, in sample
    real_model, positive_copy, negative_copy, noise_mask, models = prepare_sampling(model, noise.shape, positive, negative, noise_mask)
  File "C:\Users\theim\Projets\AI\Fooocus\backend\headless\fcbh\sample.py", line 86, in prepare_sampling
    fcbh.model_management.load_models_gpu([model] + models, model.memory_required(noise_shape) + inference_memory)
  File "C:\Users\theim\Projets\AI\Fooocus\modules\patch.py", line 494, in patched_load_models_gpu
    y = fcbh.model_management.load_models_gpu_origin(*args, **kwargs)
  File "C:\Users\theim\Projets\AI\Fooocus\backend\headless\fcbh\model_management.py", line 410, in load_models_gpu
    cur_loaded_model = loaded_model.model_load(lowvram_model_memory)
  File "C:\Users\theim\Projets\AI\Fooocus\backend\headless\fcbh\model_management.py", line 298, in model_load
    accelerate.dispatch_model(self.real_model, device_map=device_map, main_device=self.device)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\accelerate\big_modeling.py", line 371, in dispatch_model
    attach_align_device_hook_on_blocks(
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\accelerate\hooks.py", line 536, in attach_align_device_hook_on_blocks
    attach_align_device_hook_on_blocks(
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\accelerate\hooks.py", line 536, in attach_align_device_hook_on_blocks
    attach_align_device_hook_on_blocks(
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\accelerate\hooks.py", line 506, in attach_align_device_hook_on_blocks
    add_hook_to_module(module, hook)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\accelerate\hooks.py", line 155, in add_hook_to_module
    module = hook.init_hook(module)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\accelerate\hooks.py", line 253, in init_hook
    set_module_tensor_to_device(module, name, self.execution_device)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\accelerate\utils\modeling.py", line 292, in set_module_tensor_to_device
    new_value = old_value.to(device)
  File "C:\Users\theim\Projets\AI\python_embeded\lib\site-packages\torch\cuda\__init__.py", line 239, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
Total time: 169.69 seconds

My bat file looks like this : 

.\python_embeded\python.exe -m pip uninstall torch torchvision torchaudio torchtext functorch xformers -y
.\python_embeded\python.exe -m pip install torch-directml
.\python_embeded\python.exe -s Fooocus\entry_with_update.py --directml --lowvram 
pause

I also had to modify python_embeded\Lib\site-packages\torchsde_brownian\brownian_interval.py generator = torch.Generator(device).manual_seed(int(seed)) to generator = torch.Generator("cpu").manual_seed(int(seed)) in order for the code to run

ljnath commented 10 months ago

+1 to this problem. I am also facing the same.

jferrettiboke commented 10 months ago

I'm running on macOS, and passing --cpu works for me. However, slow AF. python entry_with_update.py --cpu

kespeart commented 10 months ago

python entry_with_update.py --cpu

@jferrettiboke this also works for me on macOS but like you said it is very slow

KitsonBroadhurst commented 10 months ago

Is there any other solution suggested for mac?

properboraz commented 10 months ago

Is there any other solution suggested for mac?

Same here, passing --cpu makes it work but it's extremely slow - are there any other solutions or workarounds for this?

EDIT: It seems like it's going to work but then gets stuck on 'Waiting for task to start' and then gives a connection timed out error. Anyone else getting this?

kespeart commented 10 months ago

@properboraz Facing the same issue

juanelo97 commented 10 months ago

Facing the same issue with AMD video card. Torch not compiled with CUDA enabled

lllyasviel commented 10 months ago

Hi if you see "Torch not compiled with CUDA enabled" on any AMD device, then it means you have not followed official installation guide for AMD.

See also:

https://github.com/lllyasviel/Fooocus#windowsamd-gpus

https://github.com/lllyasviel/Fooocus/blob/main/troubleshoot.md

juanelo97 commented 10 months ago

@lllyasviel i did, the only thing different is that I had to add some flags (args: --directml 0) because If I don't add those options, fooocus wont recognize my video card

PradHolla commented 9 months ago

@cufarvid should I have cuda installed from nvidia before running the command?

rachelcenter commented 1 month ago

I tried to install manually on Windows but keep getting the following during startup . . . AssertionError: Torch not compiled with CUDA enabled I have CUDA toolkit installed. Tried Automatic1111 and it works fine.

I was able to resolve same issue by forcibly reinstalling just the torch files via pytorch's instructions (https://pytorch.org/get-started/locally/) -

python -m pip install --upgrade --force-reinstall --no-cache-dir --no-deps torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

D:\ComfyUI_windows_portable\python_embeded\Scripts>python -m pip install --upgrade --force-reinstall --no-cache-dir --no-deps torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 Defaulting to user installation because normal site-packages is not writeable Looking in indexes: https://download.pytorch.org/whl/cu117 ERROR: Could not find a version that satisfies the requirement torch (from versions: none) ERROR: No matching distribution found for torch

D:\ComfyUI_windows_portable\python_embeded\Scripts>

guhenning commented 1 month ago

I tried to install manually on Windows but keep getting the following during startup . . . AssertionError: Torch not compiled with CUDA enabled I have CUDA toolkit installed. Tried Automatic1111 and it works fine.

I was able to resolve same issue by forcibly reinstalling just the torch files via pytorch's instructions (https://pytorch.org/get-started/locally/) - python -m pip install --upgrade --force-reinstall --no-cache-dir --no-deps torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

D:\ComfyUI_windows_portable\python_embeded\Scripts>python -m pip install --upgrade --force-reinstall --no-cache-dir --no-deps torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 Defaulting to user installation because normal site-packages is not writeable Looking in indexes: https://download.pytorch.org/whl/cu117 ERROR: Could not find a version that satisfies the requirement torch (from versions: none) ERROR: No matching distribution found for torch

D:\ComfyUI_windows_portable\python_embeded\Scripts>

Try this as mentioned by @cufarvid

python -m pip install --upgrade --force-reinstall --no-cache-dir --no-deps torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

This worked for me too.