[Bug]: Creating SDXL model fails in sd_to_diff.py, line 157

araleza commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

Hi, I've installed the SDXL branch of sd_dreambooth_extension, and I'm trying to create an SDXL model. But when I click 'Create Model', I get this error output:

Extracting config from /home/zzz/Documents/Dev/sdxl/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/../configs/v1-training-default.yaml
Extracting checkpoint from /home/zzz/Documents/Dev/sdxl/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors
Something went wrong, removing model directory
Traceback (most recent call last):
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/sd_to_diff.py", line 157, in extract_checkpoint
    pipe = download_from_original_stable_diffusion_ckpt(
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/venv/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 1370, in download_from_original_stable_diffusion_ckpt
    set_module_tensor_to_device(unet, param_name, "cpu", value=param)
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/venv/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 255, in set_module_tensor_to_device
    new_module = getattr(module, split)
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'ModuleList' object has no attribute '1'
Couldn't find /home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/models/dreambooth/morvface/working/unet
Unable to extract checkpoint!
Duration: 00:00:00

Steps to reproduce the problem

The source checkpoint I've selected is:

sd_xl_base_1.0.safetensors [31e35c80fc]

and I've chosen Model Type as SDXL.

I'm able to use txt2img with this same SDXL checkpoint immediately before this, so my SDXL base model file is available and working fine.

Commit and libraries

Starting at Initializing Dreambooth and ending several lines below at [+] bitsandbytes version 0.35.4 installed..

Initializing Dreambooth Dreambooth revision: cf086c536b141fc522ff11f6cffc8b7b12da04b9 Successfully installed fastapi-0.94.1 [+] xformers version 0.0.20 installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.19.3 installed. [+] transformers version 4.30.2 installed. [+] bitsandbytes version 0.35.4 installed.

Command Line Arguments

I launch with:

python launch.py --xformers

Console logs

Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
Version: v1.6.0-RC-30-g63d3150d
Commit hash: 63d3150dc4f5c4452a4a385329eb8954f53d6451
Installing requirements
If submitting an issue on github, please provide the full startup log for debugging purposes.

Initializing Dreambooth
Dreambooth revision: cf086c536b141fc522ff11f6cffc8b7b12da04b9
Successfully installed fastapi-0.94.1
[+] xformers version 0.0.20 installed.
[+] torch version 2.0.1+cu118 installed.
[+] torchvision version 0.15.2+cu118 installed.
[+] accelerate version 0.21.0 installed.
[+] diffusers version 0.19.3 installed.
[+] transformers version 4.30.2 installed.
[+] bitsandbytes version 0.35.4 installed.

Launching Web UI with arguments: --xformers
Loading weights [31e35c80fc] from /home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors
Creating model from config: /home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/repositories/generative-models/configs/inference/sd_xl_base.yaml
Model loaded in 15.3s (load weights from disk: 0.4s, create model: 0.2s, apply weights to model: 12.6s, calculate empty prompt: 1.9s).
Applying attention optimization: xformers... done.
/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py:257: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  with gr.Row().style(equal_height=False):
/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py:1047: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  db_gallery = gr.Gallery(
/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py:1047: GradioDeprecationWarning: The 'grid' parameter will be deprecated. Please use 'columns' in the constructor instead.
  db_gallery = gr.Gallery(
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 34.1s (prepare environment: 10.2s, import torch: 1.2s, import gradio: 0.4s, setup paths: 0.5s, initialize shared: 0.1s, other imports: 0.5s, load scripts: 15.8s, create ui: 0.3s, gradio launch: 5.0s).
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:05<00:00,  6.71it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:04<00:00,  7.06it/s]
==========================================================================================███████████████████████████████████████| 70/70 [00:10<00:00,  7.04it/s]
A tensor with all NaNs was produced in VAE.
Web UI will now convert VAE into 32-bit float and retry.
To disable this behavior, disable the 'Automatically revert VAE to 32-bit floats' setting.
To always start with 32-bit VAE, use --no-half-vae commandline flag.
==========================================================================================
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 70/70 [00:11<00:00,  6.36it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 70/70 [00:11<00:00,  7.04it/s]

Extracting config from /home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/../configs/v1-training-unfrozen.yaml
Extracting checkpoint from /home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors
Something went wrong, removing model directory
Traceback (most recent call last):
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/sd_to_diff.py", line 157, in extract_checkpoint
    pipe = download_from_original_stable_diffusion_ckpt(
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/venv/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 1370, in download_from_original_stable_diffusion_ckpt
    set_module_tensor_to_device(unet, param_name, "cpu", value=param)
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/venv/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 255, in set_module_tensor_to_device
    new_module = getattr(module, split)
  File "/home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'ModuleList' object has no attribute '1'
Couldn't find /home/zmx/Documents/Dev/sdxl/stable-diffusion-webui/models/dreambooth/morvface/working/unet
Unable to extract checkpoint!
Duration: 00:00:02

Additional information

No response

araleza commented 1 year ago

Never mind this, turns out I just don't know Git command line well enough, and I was still on the main branch rather than the SDXL branch. I had incorrectly attempted to fetch this branch with:

git branch sdxl   # THIS IS WRONG
git checkout sdxl

when in fact I should have typed just

git checkout SDXL

The first version created a new local branch called sdxl that was just the same as the main branch. The second version correctly fetched the SDXL branch. The branch name is case sensitive, which is why SDXL is in all caps in the correct version.

NimaMancer commented 11 months ago

Never mind this, turns out I just don't know Git command line well enough, and I was still on the main branch rather than the SDXL branch. I had incorrectly attempted to fetch this branch with:
git branch sdxl   # THIS IS WRONG
git checkout sdxl
when in fact I should have typed just
git checkout SDXL
The first version created a new local branch called sdxl that was just the same as the main branch. The second version correctly fetched the SDXL branch. The branch name is case sensitive, which is why SDXL is in all caps in the correct version.

Hello there , I'm having the same issue and I'm a complete noob can you please tell me where you changed those git commands?

araleza commented 11 months ago

I didn't change them, I typed them into a command line prompt. Change directory in your command prompt window to where your git checkout of sd_dreambooth_extension is on your hard disk, and then type 'git checkout SDXL'. And then maybe follow up with 'git pull', I can't remember if you need to type that or not, so best just to do it.

d8ahazard / sd_dreambooth_extension