Running on Apple Silicon

[fxbeaulieu]

Do you know if this should be able to run on Macs with Apple Silicon chips or do I need CUDA ?

WebUI is being launched with the following settings : export COMMANDLINE_ARGS="--opt-sub-quad-attention --skip-torch-cuda-test --upcast-sampling --no-half-vae --use-cpu interogate --listen --port 64640 --api --enable-insecure-extension-access"

When I activate the Refiner extension in txt2img, I get the following output in the shell at some point during the generation :

 75%|████████████████████████████████▎          | 12/16 [02:10<00:43, 10.89s/it]
*** Error completing request
    Traceback (most recent call last):
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/call_queue.py", line 58, in f
        res = list(func(*args, **kwargs))
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/call_queue.py", line 37, in f
        res = func(*args, **kwargs)
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/txt2img.py", line 62, in txt2img
        processed = processing.process_images(p)
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/processing.py", line 677, in process_images
        res = process_images_inner(p)
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/processing.py", line 794, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/processing.py", line 1054, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 464, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 303, in launch_sampling
        return func()
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 464, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
      File "/Users/fxbeaulieu/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "/Users/fxbeaulieu/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "/Users/fxbeaulieu/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "/Users/fxbeaulieu/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 171, in forward
        uncond = pad_cond(uncond, num_repeats, empty)
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 74, in pad_cond
        tensor['crossattn'] = pad_cond(tensor['crossattn'], repeats, empty)
      File "/Users/fxbeaulieu/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 72, in pad_cond
        return torch.cat([tensor, empty.repeat((tensor.shape[0], repeats, 1))], axis=1)
    RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 1280 but got size 2048 for tensor number 1 in the list.

[fxbeaulieu]

Update

I got it to work by editing refiner.py and replacing every mention of "cuda" by "mps".

I am not good enough with Python to write the proper fix myself, but I guess you would need to somehow verify the device to use at the start of the script and then use the result of that check to load the model.

[wcde]

I replaced 'cuda' with devices.device, it should now use the correct device, whatever it is.

[fxbeaulieu]

Works A1 ! Thanks again man 💯