Extension fails to load

[ShadoWxShinigamI]

OS :- Ubuntu 22.04 Python :- 3.10.6 A1111 Version :- 1.6.0 Model Loaded :- SDXL-Base

Getting the error :- AttributeError: 'RegionDiffusionXL' object has no attribute 'config' Here is a log for the same :-

################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye)
################################################################

################################################################
Running on ubuntu user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Launching launch.py...
################################################################
Using TCMalloc: libtcmalloc_minimal.so.4
Python 3.10.6 (main, Oct 24 2022, 16:07:47) [GCC 11.2.0]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Faceswaplab : Use GPU requirements
Checking faceswaplab requirements
0.002868067007511854
Installing requirements for Rich-Text-to-Image
Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/adetailer':
Already up to date.

Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/deforum-for-automatic1111-webui':
Already up to date.

Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/sd-webui-faceswaplab':
Already up to date.

Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/sd-webui-rich-text':
Already up to date.

Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/sd-webui-agent-scheduler':
Already up to date.

Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/sd-webui-infinite-image-browsing':
Already up to date.

Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/sd-dynamic-prompts':
Already up to date.

Pulled changes for repository in '/home/ubuntu/repos/auto/extensions/sd-webui-controlnet':
Already up to date.

Launching Web UI with arguments: --listen --xformers --api --disable-safe-unpickle --update-all-extensions --enable-insecure-extension-access
[-] ADetailer initialized. version: 23.9.3, num models: 9
2023-09-26 08:00:38,284 - ControlNet - INFO - ControlNet v1.1.410
ControlNet preprocessor location: /home/ubuntu/repos/auto/extensions/sd-webui-controlnet/annotator/downloads
2023-09-26 08:00:38,383 - ControlNet - INFO - ControlNet v1.1.410
Loading weights [e6bb9ea85b] from /home/ubuntu/repos/auto/models/Stable-diffusion/XL/sdxl-base.safetensors
*Deforum ControlNet support: enabled*
[INFO] loading stable diffusion stabilityai/stable-diffusion-xl-base-1.0 ... attempt 0
Creating model from config: /home/ubuntu/repos/auto/repositories/generative-models/configs/inference/sd_xl_base.yaml
The config attributes {'force_upcast': True} were passed to AutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
Loading VAE weights specified in settings: /home/ubuntu/repos/auto/models/VAE/sdxl_vae.safetensors
Applying attention optimization: xformers... done.
Model loaded in 3.6s (load weights from disk: 1.4s, create model: 0.5s, apply weights to model: 1.3s).
[INFO] loading stable diffusion stabilityai/stable-diffusion-xl-base-1.0 ... attempt 1
The config attributes {'force_upcast': True} were passed to AutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
*** Error executing callback ui_tabs_callback for /home/ubuntu/repos/auto/extensions/sd-webui-rich-text/scripts/rich_text_on_tab.py
    Traceback (most recent call last):
      File "/home/ubuntu/repos/auto/modules/script_callbacks.py", line 166, in ui_tabs_callback
        res += c.callback() or []
      File "/home/ubuntu/repos/auto/extensions/sd-webui-rich-text/scripts/rich_text_on_tab.py", line 77, in on_ui_tabs
        richtext2img = RichText2Img()
      File "/home/ubuntu/repos/auto/extensions/sd-webui-rich-text/scripts/rich_text_on_tab.py", line 61, in __init__
        self.model = RegionDiffusionXL()
      File "/home/ubuntu/repos/auto/extensions/sd-webui-rich-text/scripts/models/region_diffusion_xl.py", line 112, in __init__
        self.vae = AutoencoderKL.from_pretrained(load_path, subfolder="vae", use_safetensors=True, variant=variant).to(device)
      File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 522, in __setattr__
        if name in self.__dict__ and hasattr(self.config, name):
      File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 137, in __getattr__
        raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
    AttributeError: 'RegionDiffusionXL' object has no attribute 'config'

---
Running on local URL:  http://0.0.0.0:7860```

[songweige]

Have you tried to reboot the WebUI? There is a known issue that we are still investigating - sometimes the model loading would fail due to using diffusers. I wonder if this is related to that.

[ShadoWxShinigamI]

Yep, even tried full OS reboot and deleting and reinstalling the venv folder

[songweige]

As I couldn't reproduce this exact error, I may need more information, which could be very helpful. I just updated the code so that the SD1.5 is loaded instead of SDXL. Can you try to update the extension and try again? This will help check if it is a model-specific issue or something else.

[ShadoWxShinigamI]

Hey sorry for the delay, was away from my PC, just tried again, and after the update, it loads fine with the 1.5 config. So i assume the issue is with the sdxl config

[ShadoWxShinigamI]

Now after it starts up, i can switch into the sdxl model and go upto the generation stage, but then during generation it crashes. (Could this be an issue with trying to run using fp16? Maybe a method to force fp16 fixed vae would solve this?) GPU - A10G - 24GB VRAM

The config attributes {'force_upcast': True} were passed to AutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
text_input {"ops":[{"insert":"A "},{"attributes":{"link":"it is a golden retriever, wearing sunglasses"},"insert":"dog"},{"insert":" sitting on a "},{"attributes":{"color":"#b6d7a8"},"insert":"pool chair"},{"insert":", in front of a beach, "},{"attributes":{"link":"golden hour lighting"},"insert":"sunset background"},{"insert":"\n"}]} 1024 1024 41 8.5 9 0.45 0.3 0.25 0.5 blurry, text, watermark
100%|███████████████████████████████████████████████████████████████████████████████████| 41/41 [00:24<00:00,  1.65it/s]
time lapses to get attention maps: 25.9494
4 4
  0%|                                                                                            | 0/41 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/home/ubuntu/repos/auto/extensions/sd-webui-rich-text/scripts/rich_text_on_tab.py", line 189, in generate
    rich_img = model.sample(region_text_prompts, negative_prompt=[negative_text],
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/repos/auto/extensions/sd-webui-rich-text/scripts/models/region_diffusion_xl.py", line 862, in sample
    imgs = self.vae.decode(latents_inp.to(dtype=torch.float32)).sample
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/diffusers/utils/accelerate_utils.py", line 46, in wrapper
    return method(self, *args, **kwargs)
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/diffusers/models/autoencoder_kl.py", line 264, in decode
    decoded = self._decode(z).sample
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/diffusers/models/autoencoder_kl.py", line 250, in _decode
    z = self.post_quant_conv(z)
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/repos/auto/extensions-builtin/Lora/networks.py", line 444, in network_Conv2d_forward
    return originals.Conv2d_forward(self, input)
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 463, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/ubuntu/repos/auto/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (float) and bias type (c10::Half) should be the same

[songweige]

Sorry for the late reply as I have been travelling this week. It seems a bit weird to me since the errors says the vae is in float16. However, it should be set to float32 here: https://github.com/songweige/sd-webui-rich-text/blob/a02b874c8771e3ebed31aff5b10f4f1934631e52/scripts/models/region_diffusion_xl.py#L923. Does this error happen only when you specify a font color?

[songweige]

I think I figured out what happened and made a fix to that! Let me know if you are still facing the issue!