search

TheLastBen / fast-stable-diffusion

fast-stable-diffusion + DreamBooth
MIT License
7.48k stars 1.3k forks source link

Xformers Google Colab ERROR #2836

Closed mcqx4 closed 3 months ago

mcqx4 commented 3 months ago

Getting Error in the last cell during the launch:

WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for: PyTorch 2.2.1+cu121 with CUDA 1201 (you have 2.3.0+cu121) Python 3.10.13 (you have 3.10.12) Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers) Memory-efficient attention, SwiGLU, sparse and more won't be available. Set XFORMERS_MORE_DETAILS=1 for more details

UPD: Reinstalling Xformers worked for me! Put this in the new cell above the "Start Stable Diffusion":

!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121

amirulhakim199514 commented 3 months ago

image I already reinstalled xformers, but when I start the stable diffusion, i encountered an error with the last line OSError: /usr/local/lib/python3.10/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

cosmicoxytocin commented 3 months ago

image I already reinstalled xformers, but when I start the stable diffusion, i encountered an error with the last line OSError: /usr/local/lib/python3.10/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

What version of torchtext do you have?

amirulhakim199514 commented 3 months ago

image I already reinstalled xformers, but when I start the stable diffusion, i encountered an error with the last line OSError: /usr/local/lib/python3.10/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

What version of torchtext do you have?

Im a noob in this with lack of programming skills fyi. How do navigate to find the version of it?

cosmicoxytocin commented 3 months ago

image I already reinstalled xformers, but when I start the stable diffusion, i encountered an error with the last line OSError: /usr/local/lib/python3.10/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

What version of torchtext do you have?

Im a noob in this with lack of programming skills fyi. How do navigate to find the version of it?

All good. In a new cell:

import torchtext
print(torchtext.__version__)

or

!pip show torchtext

Bare in mind that I don't think updating xformers is going to solve the issue for everyone. The problem seems to be with the colab team updating jax which in turn creates conflicts.

amirulhakim199514 commented 3 months ago

image I already reinstalled xformers, but when I start the stable diffusion, i encountered an error with the last line OSError: /usr/local/lib/python3.10/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

What version of torchtext do you have?

Im a noob in this with lack of programming skills fyi. How do navigate to find the version of it?

All good. In a new cell:

import torchtext
print(torchtext.__version__)

or

!pip show torchtext

Bare in mind that I don't think updating xformers is going to solve the issue for everyone. The problem seems to be with the colab team updating jax which in turn creates conflicts.

image So for now, what do I need to do from here. And thanks for helping. I'll bare that in mind that it could be a collab issue.

cosmicoxytocin commented 3 months ago

Temp fix for those still having issues

# Revert (downgrade) JAX back to 0.4.23
!pip uninstall -y jax jaxlib
!pip install jax==0.4.23 jaxlib==0.4.23

# May be redundant — reinstall Xformers
!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121

⬆ New Code-cell Before Start Stable-Diffusion cell.

amirulhakim199514 commented 3 months ago

Temp fix for those still having issues

# Revert (downgrade) JAX back to 0.4.23
!pip uninstall -y jax jaxlib
!pip install jax==0.4.23 jaxlib==0.4.23

# May be redundant — reinstall Xformers
!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121

⬆ New Code-cell Before Start Stable-Diffusion cell.

Yep, it's working now, thanks for your help. god bless you.

chano1025 commented 3 months ago

I also have same issue

WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 2.2.1+cu121 with CUDA 1201 (you have 2.3.0+cu121)
    Python  3.10.13 (you have 3.10.12)
  Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details

I try to this code before Start Stable-Diffusion.

  # Revert (downgrade) JAX back to 0.4.23
!pip uninstall -y jax jaxlib
!pip install jax==0.4.23 jaxlib==0.4.23

# May be redundant — reinstall Xformers
!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121

However, an error occurs.

*** Error completing request
*** Arguments: ('task(1v7kp0d0fnm46q4)', <gradio.routes.Request object at 0x7d13953a6b00>, 'tree', '', [], 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', 'Use same scheduler', '', '', [], 0, 20, 'DPM++ 2M', 'Automatic', False, '', 0.8, -1, False, -1, 0, 0, 0, ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=False, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[]), ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=False, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[]), ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=False, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[]), False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {}
    Traceback (most recent call last):
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/txt2img.py", line 109, in txt2img
        processed = processing.process_images(p)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 845, in process_images
        res = process_images_inner(p)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 59, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 981, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 1328, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 218, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_common.py", line 272, in launch_sampling
        return func()
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 218, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/src/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_cfg_denoiser.py", line 237, in forward
        x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/src/k-diffusion/k_diffusion/external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/src/k-diffusion/k_diffusion/external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_models_xl.py", line 44, in apply_model
        return self.model(x, t, cond)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack_utils.py", line 18, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack_utils.py", line 32, in __call__
        return self.__orig_func(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/diffusionmodules/wrappers.py", line 28, in forward
        return self.diffusion_model(
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_unet.py", line 91, in UNetModel_forward
        return original_forward(self, x, timesteps, context, *args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/diffusionmodules/openaimodel.py", line 993, in forward
        h = module(h, emb, context)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/diffusionmodules/openaimodel.py", line 100, in forward
        x = layer(x, context)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/attention.py", line 627, in forward
        x = block(x, context=context[i])
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/attention.py", line 459, in forward
        return checkpoint(
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/diffusionmodules/util.py", line 165, in checkpoint
        return CheckpointFunction.apply(func, len(inputs), *args)
      File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 598, in apply
        return super().apply(*args, **kwargs)  # type: ignore[misc]
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/diffusionmodules/util.py", line 182, in forward
        output_tensors = ctx.run_function(*ctx.input_tensors)
      File "/content/gdrive/MyDrive/sd/stablediffusion/generative-models/sgm/modules/attention.py", line 467, in _forward
        self.attn1(
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
        return forward_call(*args, **kwargs)
      File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack_optimizations.py", line 496, in xformers_attention_forward
        out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=get_xformers_flash_attention_op(q, k, v))
      File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/__init__.py", line 247, in memory_efficient_attention
        return _memory_efficient_attention(
      File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/__init__.py", line 365, in _memory_efficient_attention
        return _memory_efficient_attention_forward(
      File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/__init__.py", line 381, in _memory_efficient_attention_forward
        op = _dispatch_fw(inp, False)
      File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/dispatch.py", line 125, in _dispatch_fw
        return _run_priority_list(
      File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/dispatch.py", line 65, in _run_priority_list
        raise NotImplementedError(msg)
    NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
         query       : shape=(2, 1024, 10, 64) (torch.float16)
         key         : shape=(2, 1024, 10, 64) (torch.float16)
         value       : shape=(2, 1024, 10, 64) (torch.float16)
         attn_bias   : <class 'NoneType'>
         p           : 0.0
    `decoderF` is not supported because:
        xFormers wasn't build with CUDA support
        attn_bias type is <class 'NoneType'>
        operator wasn't built - see `python -m xformers.info` for more info
    `flshattF@0.0.0` is not supported because:
        xFormers wasn't build with CUDA support
        requires device with capability > (8, 0) but your GPU has capability (7, 5) (too old)
        operator wasn't built - see `python -m xformers.info` for more info
    `cutlassF` is not supported because:
        xFormers wasn't build with CUDA support
        operator wasn't built - see `python -m xformers.info` for more info
    `smallkF` is not supported because:
        max(query.shape[-1] != value.shape[-1]) > 32
        xFormers wasn't build with CUDA support
        dtype=torch.float16 (supported: {torch.float32})
        operator wasn't built - see `python -m xformers.info` for more info
        unsupported embed per head: 64

---
mcqx4 commented 3 months ago

[chano1025], there are two separeted methods

You have to try to use the first:

Revert (downgrade) JAX back to 0.4.23

!pip uninstall -y jax jaxlib !pip install jax==0.4.23 jaxlib==0.4.23

or the second:

May be redundant — reinstall Xformers

!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121

djwashout commented 3 months ago

I'm having the same issue. ill wait it out for a solution.

chano1025 commented 3 months ago 
 # Revert (downgrade) JAX back to 0.4.23
!pip uninstall -y jax jaxlib
!pip install jax==0.4.23 jaxlib==0.4.23

# May be redundant — reinstall Xformers
!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121

If this code cell is added right before the "Start Stable-Diffusion" cell is executed, it works correctly. The reason why this code did not work initially was because I added it below the "Connect Google Drive" cell.

cosmicoxytocin commented 3 months ago

I'm having the same issue. ill wait it out for a solution.

https://github.com/TheLastBen/fast-stable-diffusion/issues/2836#issuecomment-2123531245

This should work as a temporary fix until TheLastBen updates the dependencies.

MMO-Interactive commented 3 months ago

This "fix" doesn't work for me. I run it just before starting up the start stable diffusion code block. It takes forever to load then eventually it uses the entire 12 gigs of system ram for the Google Colab and ends the code block.

fireincairo00 commented 3 months ago

the fix works but somehow introduces a new issue: using upscaler takes forever - it somehow adds an additional upscaling cycle with 956 iterations - the first is 86 or something and was quick, then the second cycle needs to run a long time... it was not like this before the fix

Update: and it's getting wilder - after an upscale I tried to do an image2image, the Generate button somehow triggers a new upscaling cycle with 14000 sth. iteration... I had to click on the interupt on the image2image tab, then the upscaling was stopped and the actual generation of img2img started to run... very strange

MMO-Interactive commented 3 months ago

For those of you who have found this fix to work even partially which version of Google Colabs are you using? Which Runtime? Seems odd that we would all being running in google colabs but it works for some but not for others.

TheLastBen commented 3 months ago

fixed