siliconflow / onediff

OneDiff: An out-of-the-box acceleration library for diffusion models.
https://github.com/siliconflow/onediff/wiki
Apache License 2.0
1.56k stars 93 forks source link

跑 SDXL 出错 #665

Closed Tomorrowxxy closed 5 months ago

Tomorrowxxy commented 6 months ago

跑 SDXL 出错

stable-diffusion-webui git commit id

v1.8.0-RC 310d6b9075c6edb3b884bd2a41e1c17259b141e1

OneDiff git commit id

9cbc1535a7a8662cee98635b6576f2dd66df95bb

OneFlow version info

path: ['/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow'] version: 0.9.1.dev20240220+cu118 git_commit: 6621521 cmake_build_type: Release rdma: True mlir: True enterprise: False

Tomorrowxxy commented 6 months ago

执行以下命令后 cd stable-diffusion-webui && git checkout 4afaaf8 cd repositories/generative-models && git checkout 9d759324

错误信息

txt2img: Sunflower
WARNING:onediff.infer_compiler.utils.log_utils:Failed to import onediff_quant, Error message: No module named 'onediff_quant',  Advanced features cannot be used !!!
If you need unrestricted multiple resolution, quantization support or any other more advanced features, please send an email to caishenghang@oneflow.org and tell us about your use case, deployment scale and requirements.

WARNING:onediff.infer_compiler.utils.log_utils: Advanced features cannot be used !!!
If you need unrestricted multiple resolution, quantization support or any other more advanced features, please send an email to caishenghang@oneflow.org and tell us about your use case, deployment scale and requirements.

  0%|                                                                                                     | 0/20 [00:00<?, ?it/s][ERROR](GRAPH:OneflowGraph_0:OneflowGraph) building graph got error.
ERROR:onediff.infer_compiler.utils.log_utils:Exception in __call__: e=RuntimeError("\x1b[1m\x1b[38;2;255;000;000mError\x1b[0m: expected Tensor or None as element 0, but got <class 'method'>\n")
WARNING:onediff.infer_compiler.utils.log_utils:Recompile oneflow module ...
[ERROR](GRAPH:OneflowGraph_1:OneflowGraph) building graph got error.
  0%|                                                                                                     | 0/20 [00:05<?, ?it/s]
*** Error completing request
*** Arguments: ('task(erhbyafdebwug1k)', 'Sunflower', '', [], 20, 'DPM++ 2M Karras', 1, 1, 7, 1024, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x7f3ab0e34190>, 4, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, '\n                    <div style="padding: 20px; border: 1px solid #e0e0e0; border-radius: 5px; background-color: #f9f9f9;">\n                        <div style="font-size: 18px; font-weight: bold; margin-bottom: 15px; color: #31708f;">\n                            Hints Message\n                        </div>\n                        <div style="padding: 10px; border: 1px solid #31708f; border-radius: 5px; background-color: #f9f9f9;">\n                            Hints: Enterprise function is not supported on your system.\n                        </div>\n                        <p style="margin-top: 15px;">\n                            If you need Enterprise Level Support for your system or business, please send an email to \n                            <a href="mailto:business@siliconflow.com" style="color: #31708f; text-decoration: none;">business@siliconflow.com</a>.\n                            <br>\n                            Tell us about your use case, deployment scale, and requirements.\n                        </p>\n                        <p>\n                            <strong>GitHub Issue:</strong>\n                            <a href="https://github.com/siliconflow/onediff/issues" style="color: #31708f; text-decoration: none;">https://github.com/siliconflow/onediff/issues</a>\n                        </p>\n                    </div>\n                    ') {}
    Traceback (most recent call last):
      File "/opt/project/onediff/src/onediff/infer_compiler/with_oneflow_compile.py", line 185, in wrapper
        return func(self, *args, **kwargs)
      File "/opt/project/onediff/src/onediff/infer_compiler/utils/graph_management_utils.py", line 82, in wrapper
        ret = func(self, *args, **kwargs)
      File "/opt/project/onediff/src/onediff/infer_compiler/with_oneflow_compile.py", line 272, in __call__
        output = dpl_graph(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 295, in __call__
        self._compile(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 861, in _compile
        return self._dynamic_input_graph_cache._compile(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/cache.py", line 121, in _compile
        return graph._compile(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 865, in _compile
        return self._compile_new(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 884, in _compile_new
        _, eager_outputs = self.build_graph(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 1428, in build_graph
        outputs = self.__build_graph(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 1576, in __build_graph
        outputs = self.build(*lazy_args, **lazy_kwargs)
      File "/opt/project/onediff/src/onediff/infer_compiler/with_oneflow_compile.py", line 341, in build
        return self.model(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 188, in __call__
        result = self.__block_forward(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 238, in __block_forward
        result = unbound_forward_of_module_instance(self, *args, **kwargs)
      File "/opt/project/stable-diffusion-webui/extensions/onediff_sd_webui_extensions/compile_sgm.py", line 33, in forward
        h = module(h, emb, context)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 188, in __call__
        result = self.__block_forward(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 238, in __block_forward
        result = unbound_forward_of_module_instance(self, *args, **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/generative-models/sgm/modules/diffusionmodules/openaimodel.py", line 98, in forward
        x = layer(x, emb)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 188, in __call__
        result = self.__block_forward(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 238, in __block_forward
        result = unbound_forward_of_module_instance(self, *args, **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/generative-models/sgm/modules/diffusionmodules/openaimodel.py", line 317, in forward
        return checkpoint(
      File "/opt/project/stable-diffusion-webui/repositories/generative-models/sgm/modules/diffusionmodules/util.py", line 165, in checkpoint
        return CheckpointFunction.apply(func, len(inputs), *args)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/autograd/autograd_function.py", line 65, in apply
        return AutogradFunctionBase.apply(
    oneflow._oneflow_internal.exception.RuntimeError: Error: expected Tensor or None as element 0, but got <class 'method'>

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/opt/project/stable-diffusion-webui/modules/call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "/opt/project/stable-diffusion-webui/modules/call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "/opt/project/stable-diffusion-webui/modules/txt2img.py", line 52, in txt2img
        processed = modules.scripts.scripts_txt2img.run(p, *args)
      File "/opt/project/stable-diffusion-webui/modules/scripts.py", line 601, in run
        processed = script.run(p, *script_args)
      File "/opt/project/stable-diffusion-webui/extensions/onediff_sd_webui_extensions/scripts/onediff.py", line 152, in run
        proc = process_images(p)
      File "/opt/project/stable-diffusion-webui/modules/processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "/opt/project/stable-diffusion-webui/modules/processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "/opt/project/stable-diffusion-webui/modules/processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "/opt/project/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/opt/project/stable-diffusion-webui/modules/sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "/opt/project/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/opt/project/stable-diffusion-webui/modules/sd_samplers_cfg_denoiser.py", line 169, in forward
        x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "/opt/project/stable-diffusion-webui/modules/sd_models_xl.py", line 37, in apply_model
        return self.model(x, t, cond)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/opt/project/stable-diffusion-webui/modules/sd_hijack_utils.py", line 17, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "/opt/project/stable-diffusion-webui/modules/sd_hijack_utils.py", line 28, in __call__
        return self.__orig_func(*args, **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/generative-models/sgm/modules/diffusionmodules/wrappers.py", line 28, in forward
        return self.diffusion_model(
      File "/opt/project/onediff/src/onediff/infer_compiler/utils/args_tree_util.py", line 50, in wrapper
        output = func(self, *mapped_args, **mapped_kwargs)
      File "/opt/project/onediff/src/onediff/infer_compiler/with_oneflow_compile.py", line 191, in wrapper
        return func(self, *args, **kwargs)
      File "/opt/project/onediff/src/onediff/infer_compiler/utils/graph_management_utils.py", line 82, in wrapper
        ret = func(self, *args, **kwargs)
      File "/opt/project/onediff/src/onediff/infer_compiler/with_oneflow_compile.py", line 272, in __call__
        output = dpl_graph(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 295, in __call__
        self._compile(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 861, in _compile
        return self._dynamic_input_graph_cache._compile(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/cache.py", line 121, in _compile
        return graph._compile(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 865, in _compile
        return self._compile_new(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 884, in _compile_new
        _, eager_outputs = self.build_graph(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 1428, in build_graph
        outputs = self.__build_graph(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 1576, in __build_graph
        outputs = self.build(*lazy_args, **lazy_kwargs)
      File "/opt/project/onediff/src/onediff/infer_compiler/with_oneflow_compile.py", line 341, in build
        return self.model(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 188, in __call__
        result = self.__block_forward(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 238, in __block_forward
        result = unbound_forward_of_module_instance(self, *args, **kwargs)
      File "/opt/project/stable-diffusion-webui/extensions/onediff_sd_webui_extensions/compile_sgm.py", line 33, in forward
        h = module(h, emb, context)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 188, in __call__
        result = self.__block_forward(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 238, in __block_forward
        result = unbound_forward_of_module_instance(self, *args, **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/generative-models/sgm/modules/diffusionmodules/openaimodel.py", line 98, in forward
        x = layer(x, emb)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 188, in __call__
        result = self.__block_forward(*args, **kwargs)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 238, in __block_forward
        result = unbound_forward_of_module_instance(self, *args, **kwargs)
      File "/opt/project/stable-diffusion-webui/repositories/generative-models/sgm/modules/diffusionmodules/openaimodel.py", line 317, in forward
        return checkpoint(
      File "/opt/project/stable-diffusion-webui/repositories/generative-models/sgm/modules/diffusionmodules/util.py", line 165, in checkpoint
        return CheckpointFunction.apply(func, len(inputs), *args)
      File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/autograd/autograd_function.py", line 65, in apply
        return AutogradFunctionBase.apply(
    oneflow._oneflow_internal.exception.RuntimeError: Error: expected Tensor or None as element 0, but got <class 'method'>
strint commented 5 months ago

File "/opt/miniconda3/envs/stable/lib/python3.10/site-packages/oneflow/autograd/autograd_function.py", line 65, in apply return AutogradFunctionBase.apply( oneflow._oneflow_internal.exception.RuntimeError: Error: expected Tensor or None as element 0, but got <class 'method'>

@marigoold 你那里应该处理过这个问题

marigoold commented 5 months ago

您好 @Tomorrowxxy 可以升级 onediff 到最新版本试一下

strint commented 5 months ago

pull 最新的 onediff 后,记得要更新下 onediff_sd_webui_extensions 文件夹到 stable-diffusion-webui/extensions/onediff_sd_webui_extensions

@Tomorrowxxy

Tomorrowxxy commented 5 months ago

@strint @marigoold 抱歉,之前处理了这个,这里忘了 close 了。