webui 切换模型会报错

[mir-of]

先跑sd_xl_base_1.0.safetensors，没问题。 然后底模切换到v1-5-pruned.safetensors ，报错了。

Error when switching model from sd_xl_base_1.0.safetensors to v1-5-pruned.safetensors

Reusing loaded model sd_xl_base_1.0.safetensors [31e35c80fc] to load v1-5-pruned.safetensors [1a189f0be6]
Loading weights [1a189f0be6] from /app/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned.safetensors
Creating model from config: /app/stable-diffusion-webui/configs/v1-inference.yaml
Applying attention optimization: Doggettx... done.
Model loaded in 2.3s (create model: 1.1s, apply weights to model: 1.1s).

  0%|          | 0/20 [00:00<?, ?it/s]ERROR [2023-12-19 15:31:06] - Exception in __call__: e=AssertionError('must specify y if and only if the model is class-conditional')
ERROR:ONEDIFF:Exception in __call__: e=AssertionError('must specify y if and only if the model is class-conditional')

  0%|          | 0/20 [00:00<?, ?it/s]
*** Error completing request
*** Arguments: ('task(trdkcxlud9g1t49)', 'cat', '', [], 20, 'DPM++ 2M Karras', 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x7fcf1409f0d0>, 4, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/with_oneflow_compile.py", line 165, in wrapper
        return func(self, *args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/with_oneflow_compile.py", line 235, in __call__
        output = dpl_graph(*args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 281, in __call__
        return self._dynamic_input_graph_cache(*args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/nn/graph/cache.py", line 116, in __call__
        return graph(*args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 284, in __call__
        self._compile(*args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 854, in _compile
        return self._compile_from_shared(*args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/nn/graph/graph.py", line 932, in _compile_from_shared
        outputs = self.build(*lazy_args, **lazy_kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/with_oneflow_compile.py", line 324, in build
        return self.model(*args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 188, in __call__
        result = self.__block_forward(*args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/nn/graph/proxy.py", line 238, in __block_forward
        result = unbound_forward_of_module_instance(self, *args, **kwargs)
      File "/app/stable-diffusion-webui/tmp/.mock_cache/mock_sgm_oflow_23_1702979912_381b489244a6443a88078d685f9ec358/modules/diffusionmodules/openaimodel.py", line 832, in forward
        assert (y is not None) == (
    AssertionError: must specify y if and only if the model is class-conditional

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/app/stable-diffusion-webui/modules/call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "/app/stable-diffusion-webui/modules/call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "/app/stable-diffusion-webui/modules/txt2img.py", line 52, in txt2img
        processed = modules.scripts.scripts_txt2img.run(p, *args)
      File "/app/stable-diffusion-webui/modules/scripts.py", line 601, in run
        processed = script.run(p, *script_args)
      File "/app/stable-diffusion-webui/extensions/speedup_script/scripts/onediff.py", line 197, in run
        proc = process_images(p)
      File "/app/stable-diffusion-webui/modules/processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "/app/stable-diffusion-webui/modules/processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "/app/stable-diffusion-webui/modules/processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "/app/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/app/stable-diffusion-webui/modules/sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "/app/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "/app/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/app/stable-diffusion-webui/modules/sd_samplers_cfg_denoiser.py", line 169, in forward
        x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/app/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "/app/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "/app/stable-diffusion-webui/modules/sd_hijack_utils.py", line 17, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "/app/stable-diffusion-webui/modules/sd_hijack_utils.py", line 28, in __call__
        return self.__orig_func(*args, **kwargs)
      File "/app/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 858, in apply_model
        x_recon = self.model(x_noisy, t, **cond)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/app/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 1335, in forward
        out = self.diffusion_model(x, t, context=cc)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/utils/args_tree_util.py", line 34, in wrapper
        output = func(cls, *mapped_args, **mapped_kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/with_oneflow_compile.py", line 171, in wrapper
        return func(self, *args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/with_oneflow_compile.py", line 233, in __call__
        dpl_graph = self.get_graph()
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/with_oneflow_compile.py", line 207, in get_graph
        self._deployable_module_model.oneflow_module, size, all_dynamic
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/with_oneflow_compile.py", line 31, in oneflow_module
        self._oneflow_module = torch2oflow(self._torch_module)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 38, in wrapper
        result = dispatcher(first_param, *args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/functools.py", line 889, in wrapper
        return dispatch(args[0].__class__)(*args, **kw)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 221, in _
        of_mod = of_mod_cls()
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 191, in init
        self._modules[n] = torch2oflow(m)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 38, in wrapper
        result = dispatcher(first_param, *args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/functools.py", line 889, in wrapper
        return dispatch(args[0].__class__)(*args, **kw)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 252, in _
        submod = torch2oflow(original_submod, verbose)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 38, in wrapper
        result = dispatcher(first_param, *args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/functools.py", line 889, in wrapper
        return dispatch(args[0].__class__)(*args, **kw)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 221, in _
        of_mod = of_mod_cls()
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 187, in init
        self._parameters[n] = torch2oflow(p)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 38, in wrapper
        result = dispatcher(first_param, *args, **kwargs)
      File "/root/anaconda3/envs/osd/lib/python3.10/functools.py", line 889, in wrapper
        return dispatch(args[0].__class__)(*args, **kw)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/onediff/infer_compiler/transform/builtin_transform.py", line 261, in _
        data = flow.utils.tensor.from_torch(mod.data)
      File "/root/anaconda3/envs/osd/lib/python3.10/site-packages/oneflow/utils/tensor/from_or_to_torch_tensor.py", line 65, in from_torch
        return flow.from_dlpack(torch.to_dlpack(torch_tensor))
    RuntimeError: Cannot pack tensors on meta

oneflow version: '0.9.1.dev20231217+cu118' onediff: '0.11.0.dev'

stable diffusion webui: version: v1.6.1  •  python: 3.10.13  •  torch: 2.0.1+cu118  •  xformers: N/A  •  gradio: 3.41.2  •  checkpoint: 1a189f0be6

[strint]

@fpzh2011 let's look into this.

[zsh123abc]

大佬，如何把添加到sd中的呀

[fpzh2011]

sd-webui 的 onediff 插件目前只支持 sdxl 模型，还不支持 v1-5-pruned 等模型。我们会尽快修复这个问题。

[fpzh2011]

@mir-of sd-webui 的 onediff 插件目前已经支持 v1-5-pruned 模型，加速性能也有了提升。

[mir-of]

额，应该不是 v1-5-pruned 模型未支持的原因。现在问题仍存在，从v1-5-pruned切换到 sd_xl_base_1.0.safetensors，还是会报错。

[fpzh2011]

额，应该不是 v1-5-pruned 模型未支持的原因。现在问题仍存在，从v1-5-pruned切换到 sd_xl_base_1.0.safetensors，还是会报错。

@mir-of 您可以确认一下 sgm 的版本，参考 插件安装指南 中的如下命令：

cd repositories/generative-models && git checkout 9d759324

[strint]

No new feedback after supplying a solution, so this issue will be closed. Feel free to reopen it.

[yy-fisher]

遇到同样的情况在 v1-5-pruned 切换到 sd_xl_base_1.0.safetensors，还是会报错。并且执行到cd repositories/generative-models && git checkout 9d759324命令，期待你的回答