search

openvinotoolkit / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
250 stars 39 forks source link

[Bug]: RuntimeError: Windows not yet supported for torch.compile #49

Closed baselqt closed 8 months ago

baselqt commented 9 months ago

Is there an existing issue for this?

What happened?

it shows RuntimeError: Windows not yet supported for torch.compile when i try running inference on gpu

Steps to reproduce the problem

  1. launch the webui with arguments --skip-torch-cuda-test
  2. select the oppenvino script and choose device then load the model
  3. try generating

What should have happened?

it should generate normally

Version or Commit where the problem happens

1

What Python version are you running on ?

Python 3.10.x

What platforms do you use to access the UI ?

Windows

What device are you running WebUI on?

CPU, Other GPUs

Cross attention optimization

Automatic

What browsers do you use to access the UI ?

Microsoft Edge

Command Line Arguments

--skip-torch-cuda-test or --skip-torch-cuda-test --no-half

List of extensions

none

Console logs

Creating model from config: E:\stable diffusion\openvinosd\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
Exception in thread Thread-19 (first_time_calculation):
Traceback (most recent call last):
  File "C:\Users\basel\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\basel\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\devices.py", line 167, in first_time_calculation
    linear(x)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\extensions-builtin\Lora\networks.py", line 361, in network_Linear_forward
    return torch.nn.Linear_forward_before_network(self, input)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'
DiffusionWrapper has 859.52 M params.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 88.6s (launcher: 4.1s, import torch: 45.4s, import gradio: 6.2s, setup paths: 8.2s, other imports: 10.6s, setup codeformer: 0.5s, load scripts: 10.1s, load upscalers: 0.1s, initialize extra networks: 0.1s, create ui: 2.5s, gradio launch: 0.5s).
Applying attention optimization: InvokeAI... done.
loading stable diffusion model: RuntimeError
Traceback (most recent call last):
  File "C:\Users\basel\AppData\Local\Programs\Python\Python310\lib\threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\basel\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "C:\Users\basel\AppData\Local\Programs\Python\Python310\lib\threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\webui.py", line 318, in load_model
    shared.sd_model  # noqa: B018
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\shared.py", line 754, in sd_model
    return modules.sd_models.model_data.get_sd_model()
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_models.py", line 439, in get_sd_model
    load_model()
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_models.py", line 536, in load_model
    sd_model.cond_stage_model_empty_prompt = get_empty_cond(sd_model)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_models.py", line 460, in get_empty_cond
    return sd_model.cond_stage_model([""])
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_hijack_clip.py", line 234, in forward
    z = self.process_tokens(tokens, multipliers)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_hijack_clip.py", line 271, in process_tokens
    z = self.encode_with_transformers(tokens)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_hijack_clip.py", line 324, in encode_with_transformers
    outputs = self.wrapped.transformer(input_ids=tokens, output_hidden_states=-opts.CLIP_stop_at_last_layers)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 822, in forward
    return self.text_model(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 740, in forward
    encoder_outputs = self.encoder(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 654, in forward
    layer_outputs = encoder_layer(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 382, in forward
    hidden_states = self.layer_norm1(hidden_states)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\normalization.py", line 190, in forward
    return F.layer_norm(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 2515, in layer_norm
    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

Stable diffusion model failed to load
*** Error completing request
*** Arguments: ('task(dsju4giw4xlmlv0)', 'a picture of a flower in a dirt field', '', [], 20, 0, False, False, 1, 1, 5, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], <gradio.routes.Request object at 0x000002A0BD59B820>, 1, 'None', 'GPU', True, 'Euler', True, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0) {}
    Traceback (most recent call last):
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 58, in f
        res = list(func(*args, **kwargs))
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 37, in f
        res = func(*args, **kwargs)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\txt2img.py", line 59, in txt2img
        processed = modules.scripts.scripts_txt2img.run(p, *args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\scripts.py", line 501, in run
        processed = script.run(p, *script_args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 1085, in run
        processed = process_images_openvino(p, model_config, p.sampler_name, enable_caching, openvino_device, mode)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 828, in process_images_openvino
        or model_state.mode != mode or model_state.model_hash != shared.sd_model.sd_model_hash or model_state.cn_model != cn_model):
    AttributeError: 'NoneType' object has no attribute 'sd_model_hash'

---
*** Error completing request
*** Arguments: ('task(2pz4tirxh8mo34q)', 'a picture of a flower in a dirt field', '', [], 20, 0, False, False, 1, 1, 5, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], <gradio.routes.Request object at 0x000002A0BD598130>, 1, 'v1-inference.yaml', 'GPU', True, 'Euler', True, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0) {}
    Traceback (most recent call last):
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 58, in f
        res = list(func(*args, **kwargs))
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 37, in f
        res = func(*args, **kwargs)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\txt2img.py", line 59, in txt2img
        processed = modules.scripts.scripts_txt2img.run(p, *args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\scripts.py", line 501, in run
        processed = script.run(p, *script_args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 1085, in run
        processed = process_images_openvino(p, model_config, p.sampler_name, enable_caching, openvino_device, mode)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 828, in process_images_openvino
        or model_state.mode != mode or model_state.model_hash != shared.sd_model.sd_model_hash or model_state.cn_model != cn_model):
    AttributeError: 'NoneType' object has no attribute 'sd_model_hash'

---
*** Error completing request
*** Arguments: ('task(7d8uxjdsazmebbf)', 'a picture of a flower in a dirt field', '', [], 20, 0, False, False, 1, 1, 5, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], <gradio.routes.Request object at 0x000002A192F35C90>, 1, 'v1-inference.yaml', 'CPU', True, 'Euler', True, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0) {}
    Traceback (most recent call last):
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 58, in f
        res = list(func(*args, **kwargs))
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 37, in f
        res = func(*args, **kwargs)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\txt2img.py", line 59, in txt2img
        processed = modules.scripts.scripts_txt2img.run(p, *args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\scripts.py", line 501, in run
        processed = script.run(p, *script_args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 1085, in run
        processed = process_images_openvino(p, model_config, p.sampler_name, enable_caching, openvino_device, mode)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 828, in process_images_openvino
        or model_state.mode != mode or model_state.model_hash != shared.sd_model.sd_model_hash or model_state.cn_model != cn_model):
    AttributeError: 'NoneType' object has no attribute 'sd_model_hash'

---
Calculating sha256 for E:\stable diffusion\openvinosd\stable-diffusion-webui\models\Stable-diffusion\junior.safetensors: 84d76a0328ee5a1a2a1d466ba7478ab59183e3fea385441528aa4c5d567fd43e
Loading weights [84d76a0328] from E:\stable diffusion\openvinosd\stable-diffusion-webui\models\Stable-diffusion\junior.safetensors
Creating model from config: E:\stable diffusion\openvinosd\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying attention optimization: InvokeAI... done.
changing setting sd_model_checkpoint to junior.safetensors: RuntimeError
Traceback (most recent call last):
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\shared.py", line 633, in set
    self.data_labels[key].onchange()
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\webui.py", line 238, in <lambda>
    shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights()), call=False)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_models.py", line 578, in reload_model_weights
    load_model(checkpoint_info, already_loaded_state_dict=state_dict)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_models.py", line 536, in load_model
    sd_model.cond_stage_model_empty_prompt = get_empty_cond(sd_model)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_models.py", line 460, in get_empty_cond
    return sd_model.cond_stage_model([""])
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_hijack_clip.py", line 234, in forward
    z = self.process_tokens(tokens, multipliers)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_hijack_clip.py", line 271, in process_tokens
    z = self.encode_with_transformers(tokens)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\sd_hijack_clip.py", line 324, in encode_with_transformers
    outputs = self.wrapped.transformer(input_ids=tokens, output_hidden_states=-opts.CLIP_stop_at_last_layers)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 822, in forward
    return self.text_model(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 740, in forward
    encoder_outputs = self.encoder(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 654, in forward
    layer_outputs = encoder_layer(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\transformers\models\clip\modeling_clip.py", line 382, in forward
    hidden_states = self.layer_norm1(hidden_states)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\normalization.py", line 190, in forward
    return F.layer_norm(
  File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 2515, in layer_norm
    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

Loading weights [84d76a0328] from E:\stable diffusion\openvinosd\stable-diffusion-webui\models\Stable-diffusion\junior.safetensors
OpenVINO Script:  created model from config : E:\stable diffusion\openvinosd\stable-diffusion-webui\configs\v1-inference.yaml
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
*** Error completing request
*** Arguments: ('task(rka80f0gm85rjcv)', 'a picture of a flower in a dirt field', '', [], 20, 0, False, False, 1, 1, 5, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], <gradio.routes.Request object at 0x000002A0FC42A2C0>, 1, 'v1-inference.yaml', 'GPU', True, 'Euler', True, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0) {}
    Traceback (most recent call last):
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 58, in f
        res = list(func(*args, **kwargs))
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\call_queue.py", line 37, in f
        res = func(*args, **kwargs)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\txt2img.py", line 59, in txt2img
        processed = modules.scripts.scripts_txt2img.run(p, *args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\modules\scripts.py", line 501, in run
        processed = script.run(p, *script_args)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 1085, in run
        processed = process_images_openvino(p, model_config, p.sampler_name, enable_caching, openvino_device, mode)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 838, in process_images_openvino
        shared.sd_diffusers_model = get_diffusers_sd_model(model_config, sampler_name, enable_caching, openvino_device, mode)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\scripts\openvino_accelerate.py", line 625, in get_diffusers_sd_model
        sd_model.unet = torch.compile(sd_model.unet, backend="openvino_fx")
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\__init__.py", line 1441, in compile
        return torch._dynamo.optimize(backend=backend, nopython=fullgraph, dynamic=dynamic, disable=disable)(model)
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\_dynamo\eval_frame.py", line 413, in optimize
        check_if_dynamo_supported()
      File "E:\stable diffusion\openvinosd\stable-diffusion-webui\venv\lib\site-packages\torch\_dynamo\eval_frame.py", line 375, in check_if_dynamo_supported
        raise RuntimeError("Windows not yet supported for torch.compile")
    RuntimeError: Windows not yet supported for torch.compile

Additional information

No response

muqiann2022 commented 9 months ago

same issue

oh, i didn't check the installation page carefully, there's a "torch-install.bat" at the root folder could use, and it works, no error.

baselqt commented 9 months ago

now it works thank you