search

anapnoe / stable-diffusion-webui-ux

Stable Diffusion web UI UX
GNU Affero General Public License v3.0
978 stars 59 forks source link

[Feature Request]: PR "--upcast-sampling support for CUDA" is missing from this fork #62

Closed cyrusfirheir closed 1 year ago

cyrusfirheir commented 1 year ago

Is there an existing issue for this?

What happened?

Using XFormers and Upcast Sampling optimizations simultaneously results in an error. In the original A111 repository, there is a PR which fixes this. "--upcast-sampling support for CUDA".

It is missing from this fork. Merging it would be appreciated.

Steps to reproduce the problem

  1. Have both --xformers and --upcast-sampling enabled.
  2. Generate image.

What should have happened?

Normal image generation.

Commit where the problem happens

https://github.com/anapnoe/stable-diffusion-webui-ux/commit/4540d4f8ff4226eb4820059a3a52d60c72149061

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--xformers --upcast-sampling

List of extensions

No

Console logs

venv "F:\StableDiffusion\stable-diffusion-webui-ux\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 2ac62ce241396939c387a221e20b0a7a8c399b6f
Installing requirements for Web UI

Launching Web UI with arguments: --api --xformers --upcast-sampling

Loading weights [a60cfaa90d] from models\Stable-diffusion\model.ckpt
Creating model from config: F:\StableDiffusion\stable-diffusion-webui-ux\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: models/VAE\kl-f8-anime2.ckpt
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(0):
Model loaded in 3.9s (create model: 0.5s, apply weights to model: 0.9s, apply half(): 0.3s, load VAE: 0.3s, move model to device: 0.7s, load textual inversion embeddings: 1.1s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 20.2s (import gradio: 2.8s, import ldm: 3.1s, other imports: 1.9s, list extensions: 1.6s, setup codeformer: 0.1s, load scripts: 2.8s, load SD checkpoint: 4.4s, create ui: 3.2s, gradio launch: 0.2s).

Traceback (most recent call last):                                                                                      
  File "/tmp/stable-diffusion-webui/modules/call_queue.py", line 56, in f                                               
    res = list(func(*args, **kwargs))                                                                                   
  File "/tmp/stable-diffusion-webui/modules/call_queue.py", line 37, in f                                               
    res = func(*args, **kwargs)                                                                                         
  File "/tmp/stable-diffusion-webui/modules/txt2img.py", line 59, in txt2img                                            
    processed = process_images(p)                                                                                       
  File "/tmp/stable-diffusion-webui/modules/processing.py", line 486, in process_images                                 
    res = process_images_inner(p)                                                                                       
  File "/tmp/stable-diffusion-webui/modules/processing.py", line 678, in process_images_inner                           
    samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_stren
gth=p.subseed_strength, prompts=prompts)                                                                                
  File "/tmp/stable-diffusion-webui/modules/processing.py", line 892, in sample                                         
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_ima
ge_conditioning(x))                                                                                                     
  File "/tmp/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 353, in sample                             
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={                        
  File "/tmp/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 229, in launch_sampling                    
    return func()                                                                                                       
  File "/tmp/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 353, in <lambda>                           
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={                        
  File "/usr/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context                    
    return func(*args, **kwargs)                                                                                        
  File "/tmp/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 145, in sample_euler_ancestr
al                                                                                                                      
    denoised = model(x, sigmas[i] * s_in, **extra_args)                                                                 
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl                         
    return forward_call(*args, **kwargs)                                                                                
  File "/tmp/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 121, in forward                            
    x_out = self.inner_model(x_in, sigma_in, cond={"c_crossattn": [cond_in], "c_concat": [image_cond_in]})              
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl                         
    return forward_call(*args, **kwargs)                                                                                
  File "/tmp/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 167, in forward
    return self.get_v(input * c_in, self.sigma_to_t(sigma), **kwargs) * c_out + input * c_skip
  File "/tmp/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 177, in get_v
    return self.inner_model.apply_model(x, t, cond)
  File "/tmp/stable-diffusion-webui/modules/sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "/tmp/stable-diffusion-webui/modules/sd_hijack_utils.py", line 26, in __call__
    return self.__sub_func(self.__orig_func, *args, **kwargs)
  File "/tmp/stable-diffusion-webui/modules/sd_hijack_unet.py", line 45, in apply_model
    return orig_func(self, x_noisy.to(devices.dtype_unet), t.to(devices.dtype_unet), cond, **kwargs).float()
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 858, 
in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
    out = self.diffusion_model(x, t, context=cc)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 778, in forward
    h = module(h, emb, context)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 82, in forward
    x = layer(x, emb)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 249, in forward
    return checkpoint(
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/util.py", line 114, in checkpoint
    return CheckpointFunction.apply(func, len(inputs), *args)
  File "/usr/lib/python3.10/site-packages/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/util.py", line 129, in forward
    output_tensors = ctx.run_function(*ctx.input_tensors)
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 262, in _forward
    h = self.in_layers(x)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
    input = module(input)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/tmp/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/util.py", line 219, in forward
    return super().forward(x.float()).type(x.dtype)
  File "/usr/lib/python3.10/site-packages/torch/nn/modules/normalization.py", line 273, in forward
    return F.group_norm(
  File "/usr/lib/python3.10/site-packages/torch/nn/functional.py", line 2530, in group_norm
    return torch.group_norm(input, num_groups, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of Float

Additional information

No response

anapnoe commented 1 year ago

use the dev branch if you want the latest features