[Bug]: Can't create models in Dreambooth

[isodugu]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I cannot create any models into any checkpoints because of this problem please help ive been trying for literally days

i've updated pip

i've installed xformers

İ've updated cuda

OutputErrorMissing model directory, removing model: C:\AI\stable-diffusion-webui\models\dreambooth\modelname\working\vae

Steps to reproduce the problem

1. Go to .... Dreambooth
2. Press ....Create model
3. ...OutputErrorMissing model directory, removing model: C:\AI\stable-diffusion-webui\models\dreambooth\modelname\working\vae

Commit and libraries

Starting at Initializing Dreambooth and ending several lines below at [+] bitsandbytes version 0.35.4 installed..

Initializing Dreambooth Dreambooth revision: b4053defa6ae018b2ea56ac243aa55063f76fe0e Successfully installed accelerate-0.21.0 fastapi-0.94.1 gitpython-3.1.32 transformers-4.30.2

Does your project take forever to startup? Repetitive dependency installation may be the reason. Automatic1111's base project sets strict requirements on outdated dependencies. If an extension is using a newer version, the dependency is uninstalled and reinstalled twice every startup.

[+] xformers version 0.0.20 installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.19.3 installed. [+] transformers version 4.30.2 installed. [+] bitsandbytes version 0.35.4 installed.

Launching Web UI with arguments: No module 'xformers'. Proceeding without it. [2023-08-17 02:06:02,820][DEBUG][git.cmd] - Popen(['git', 'version'], cwd=C:\AI\stable-diffusion-webui, universal_newlines=False, shell=None, istream=None) [2023-08-17 02:06:02,857][DEBUG][git.cmd] - Popen(['git', 'version'], cwd=C:\AI\stable-diffusion-webui, universal_newlines=False, shell=None, istream=None) Loading weights [cc6cb27103] from C:\AI\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.ckpt Creating model from config: C:\AI\stable-diffusion-webui\configs\v1-inference.yaml LatentDiffusion: Running in eps-prediction mode DiffusionWrapper has 859.52 M params. Model loaded in 11.1s (load weights from disk: 4.5s, create model: 0.7s, apply weights to model: 0.9s, apply half(): 1.0s, move model to device: 1.1s, load textual inversion embeddings: 1.8s, calculate empty prompt: 1.1s). [2023-08-17 02:06:16,949][DEBUG][api.py] - SD-Webui API layer loaded Applying attention optimization: Doggettx... done. [2023-08-17 02:06:17,536][DEBUG][markdown_it.rules_block.code] - entering code: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,536][DEBUG][markdown_it.rules_block.fence] - entering fence: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,538][DEBUG][markdown_it.rules_block.blockquote] - entering blockquote: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,538][DEBUG][markdown_it.rules_block.hr] - entering hr: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,539][DEBUG][markdown_it.rules_block.list] - entering list: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,539][DEBUG][markdown_it.rules_block.code] - entering code: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,540][DEBUG][markdown_it.rules_block.fence] - entering fence: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,540][DEBUG][markdown_it.rules_block.blockquote] - entering blockquote: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,541][DEBUG][markdown_it.rules_block.hr] - entering hr: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,541][DEBUG][markdown_it.rules_block.list] - entering list: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,542][DEBUG][markdown_it.rules_block.reference] - entering reference: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,542][DEBUG][markdown_it.rules_block.html_block] - entering html_block: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,542][DEBUG][markdown_it.rules_block.heading] - entering heading: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,543][DEBUG][markdown_it.rules_block.lheading] - entering lheading: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,543][DEBUG][markdown_it.rules_block.paragraph] - entering paragraph: StateBlock(line=0,level=2,tokens=2), 0, 1, False [2023-08-17 02:06:17,620][DEBUG][markdown_it.rules_block.code] - entering code: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,620][DEBUG][markdown_it.rules_block.fence] - entering fence: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,621][DEBUG][markdown_it.rules_block.blockquote] - entering blockquote: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,622][DEBUG][markdown_it.rules_block.hr] - entering hr: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,622][DEBUG][markdown_it.rules_block.list] - entering list: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,623][DEBUG][markdown_it.rules_block.reference] - entering reference: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,623][DEBUG][markdown_it.rules_block.html_block] - entering html_block: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,624][DEBUG][markdown_it.rules_block.heading] - entering heading: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,624][DEBUG][markdown_it.rules_block.lheading] - entering lheading: StateBlock(line=0,level=0,tokens=0), 0, 1, False [2023-08-17 02:06:17,624][DEBUG][markdown_it.rules_block.paragraph] - entering paragraph: StateBlock(line=0,level=0,tokens=0), 0, 1, False CUDA SETUP: Loading binary C:\AI\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cudaall.dll...

Command Line Arguments

@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS= --xformers
git pull 
call webui.bat

Console logs

Loading model from checkpoint.
Loading ckpt...
Pred and size are epsilon and 512, using config: C:\AI\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\..\configs\v1-training-default.yaml
v1 model loaded.
Trying to load: C:\AI\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\..\configs\v1-training-default.yaml
Converting unet...
[2023-08-16 23:50:43,177][DEBUG][dreambooth.dataclasses.db_config] - Saving to C:\AI\stable-diffusion-webui\models\dreambooth\modelname
Converting vae...
Exception setting up output: Error(s) in loading state_dict for AutoencoderKL:
        Missing key(s) in state_dict: "encoder.mid_block.attentions.0.to_q.weight", "encoder.mid_block.attentions.0.to_q.bias", "encoder.mid_block.attentions.0.to_k.weight", "encoder.mid_block.attentions.0.to_k.bias", "encoder.mid_block.attentions.0.to_v.weight", "encoder.mid_block.attentions.0.to_v.bias", "encoder.mid_block.attentions.0.to_out.0.weight", "encoder.mid_block.attentions.0.to_out.0.bias", "decoder.mid_block.attentions.0.to_q.weight", "decoder.mid_block.attentions.0.to_q.bias", "decoder.mid_block.attentions.0.to_k.weight", "decoder.mid_block.attentions.0.to_k.bias", "decoder.mid_block.attentions.0.to_v.weight", "decoder.mid_block.attentions.0.to_v.bias", "decoder.mid_block.attentions.0.to_out.0.weight", "decoder.mid_block.attentions.0.to_out.0.bias".
        Unexpected key(s) in state_dict: "encoder.mid_block.attentions.0.query.weight", "encoder.mid_block.attentions.0.query.bias", "encoder.mid_block.attentions.0.key.weight", "encoder.mid_block.attentions.0.key.bias", "encoder.mid_block.attentions.0.value.weight", "encoder.mid_block.attentions.0.value.bias", "encoder.mid_block.attentions.0.proj_attn.weight", "encoder.mid_block.attentions.0.proj_attn.bias", "decoder.mid_block.attentions.0.query.weight", "decoder.mid_block.attentions.0.query.bias", "decoder.mid_block.attentions.0.key.weight", "decoder.mid_block.attentions.0.key.bias", "decoder.mid_block.attentions.0.value.weight", "decoder.mid_block.attentions.0.value.bias", "decoder.mid_block.attentions.0.proj_attn.weight", "decoder.mid_block.attentions.0.proj_attn.bias".
Traceback (most recent call last):
  File "C:\AI\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\sd_to_diff.py", line 1278, in extract_checkpoint
    vae.load_state_dict(converted_vae_checkpoint)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for AutoencoderKL:
        Missing key(s) in state_dict: "encoder.mid_block.attentions.0.to_q.weight", "encoder.mid_block.attentions.0.to_q.bias", "encoder.mid_block.attentions.0.to_k.weight", "encoder.mid_block.attentions.0.to_k.bias", "encoder.mid_block.attentions.0.to_v.weight", "encoder.mid_block.attentions.0.to_v.bias", "encoder.mid_block.attentions.0.to_out.0.weight", "encoder.mid_block.attentions.0.to_out.0.bias", "decoder.mid_block.attentions.0.to_q.weight", "decoder.mid_block.attentions.0.to_q.bias", "decoder.mid_block.attentions.0.to_k.weight", "decoder.mid_block.attentions.0.to_k.bias", "decoder.mid_block.attentions.0.to_v.weight", "decoder.mid_block.attentions.0.to_v.bias", "decoder.mid_block.attentions.0.to_out.0.weight", "decoder.mid_block.attentions.0.to_out.0.bias".
        Unexpected key(s) in state_dict: "encoder.mid_block.attentions.0.query.weight", "encoder.mid_block.attentions.0.query.bias", "encoder.mid_block.attentions.0.key.weight", "encoder.mid_block.attentions.0.key.bias", "encoder.mid_block.attentions.0.value.weight", "encoder.mid_block.attentions.0.value.bias", "encoder.mid_block.attentions.0.proj_attn.weight", "encoder.mid_block.attentions.0.proj_attn.bias", "decoder.mid_block.attentions.0.query.weight", "decoder.mid_block.attentions.0.query.bias", "decoder.mid_block.attentions.0.key.weight", "decoder.mid_block.attentions.0.key.bias", "decoder.mid_block.attentions.0.value.weight", "decoder.mid_block.attentions.0.value.bias", "decoder.mid_block.attentions.0.proj_attn.weight", "decoder.mid_block.attentions.0.proj_attn.bias".
Missing model directory, removing model: C:\AI\stable-diffusion-webui\models\dreambooth\modelname\working\vae
Restored system models.
Duration: 00:00:17
Exception loading config: [Errno 2] No such file or directory: 'C:\\AI\\stable-diffusion-webui\\models\\dreambooth\\modelname\\db_config.json'
Traceback (most recent call last):
  File "C:\AI\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\dataclasses\db_config.py", line 415, in from_file
    with open(config_file, 'r') as openfile:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\AI\\stable-diffusion-webui\\models\\dreambooth\\modelname\\db_config.json'
Can't load config!
Traceback (most recent call last):
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 422, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1326, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1229, in postprocess_data
    self.validate_outputs(fn_index, predictions)  # type: ignore
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1204, in validate_outputs
    raise ValueError(
ValueError: An event handler (load_model_params) didn't receive enough output values (needed: 10, received: 7).
Wanted outputs:
    [html, html, html, html, html, html, html, dropdown, dropdown, html]
Received outputs:
    ["", "", "", "", "", {'visible': True, 'choices': [], 'value': '', '__type__': 'generic_update'}, "Error loading model params: 'modelname'."]

Additional information

No response

[po3na4skld]

I have similar problem.

It worked with manually installing diffusers~=0.16.1 and reloading AUTO111

[bklstheman]

I have similar problem.

It worked with manually installing diffusers~=0.16.1 and reloading AUTO111

Can you paste what you ran to do this? I'm running into the same issue. I assume its some terminal command but I could be wrong.

[jmaccall316]

The diffusers

I have similar problem. It worked with manually installing diffusers~=0.16.1 and reloading AUTO111

Can you paste what you ran to do this? I'm running into the same issue. I assume its some terminal command but I could be wrong.

Don't waste your time, I tried this method and ran into the same issue. Uninstalled diffusers, installed diffusers==0.16.1, restarted runtime, reloaded, still the same problem. The vae has been an ongoing issue. Part of the reason I switched to automatic1111 over diffusers training was because training kept breaking the vae, now sadly it's happening with this method too.

[bklstheman]

The diffusers

I have similar problem. It worked with manually installing diffusers~=0.16.1 and reloading AUTO111

Can you paste what you ran to do this? I'm running into the same issue. I assume its some terminal command but I could be wrong.

Don't waste your time, I tried this method and ran into the same issue. Uninstalled diffusers, installed diffusers==0.16.1, restarted runtime, reloaded, still the same problem. The vae has been an ongoing issue. Part of the reason I switched to automatic1111 over diffusers training was because training kept breaking the vae, now sadly it's happening with this method too.

Ah ok. Not sure if there is a rollback checkpoint that make sense to use? Seems like others are in the same boat with this.

[mcgrimm]

just fyi , i think my turnarround solution is try reinstall other version streambooth by

git checkout dc413a14379b165355502d9f65856c40a4bb5b6f

then pip install -r requirments.txt underneath the dreambooth folder.

[liangxuZhang]

This seems to be because the format read by the AutoencoderKL of diffusers is different from the format converted by the project. I added the following code at the end of the dreambooth/sd_to_diff.py line 188 conv_attn_to_linear() function and it was worked.

mid_attentions = [key for key in keys if "mid_block.attentions" in key]
convert_map = {
    "query.weight" : "to_q.weight",
    "query.bias" : "to_q.bias",
    "key.weight" : "to_k.weight",
    "key.bias" : "to_k.bias",
    "value.weight" : "to_v.weight",
    "value.bias" : "to_v.bias",
    "proj_attn.weight" : "to_out.0.weight",
    "proj_attn.bias" : "to_out.0.bias",
}
for key in keys:
    if key in mid_attentions:
        old_key = ".".join(key.split(".")[-2:])
        if old_key in convert_map:
            new_key = key.replace(old_key, convert_map[old_key])
            checkpoint[new_key] = checkpoint.pop(old_key)

[bklstheman]

just fyi , i think my turnarround solution is try reinstall other version streambooth by

git checkout dc413a1

then pip install -r requirments.txt underneath the dreambooth folder.

This is looking like it is working for me. Thanks for pointing out the commit.

[commander443]

dreambooth works for me but i just get black squares

[github-actions[bot]]

This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days

[alixiaodi]

Exception setting up output: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /openai/clip-vit-large-patch14/resolve/main/tf_model.h5 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002125AD0DE40>, 'Connection to huggingface.co timed out. (connect timeout=10)')) 提示：Python 运行时抛出了一个异常。请检查疑难解答页面。 Traceback (most recent call last): File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\connection.py", line 174, in _new_conn conn = connection.create_connection( File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\util\connection.py", line 95, in create_connection raise err File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection sock.connect(sa) TimeoutError: timed out 提示：Python 运行时抛出了一个异常。请检查疑难解答页面。

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request self._validate_conn(conn) File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\connectionpool.py", line 1042, in _validate_conn conn.connect() File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\connection.py", line 358, in connect self.sock = conn = self._new_conn() File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\connection.py", line 179, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x000002125AD0DE40>, 'Connection to huggingface.co timed out. (connect timeout=10)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\sd-webui-aki-v4\py310\lib\site-packages\requests\adapters.py", line 486, in send resp = conn.urlopen( File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen retries = retries.increment( File "D:\sd-webui-aki-v4\py310\lib\site-packages\urllib3\util\retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /openai/clip-vit-large-patch14/resolve/main/tf_model.h5 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002125AD0DE40>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\sd-webui-aki-v4\extensions\sd_dreambooth_extension\dreambooth\sd_to_diff.py", line 1290, in extract_checkpoint text_model = convert_ldm_clip_checkpoint(checkpoint) File "D:\sd-webui-aki-v4\extensions\sd_dreambooth_extension\dreambooth\sd_to_diff.py", line 622, in convert_ldm_clip_checkpoint text_model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14") File "D:\sd-webui-aki-v4\py310\lib\site-packages\transformers\modeling_utils.py", line 2472, in from_pretrained if has_file(pretrained_model_name_or_path, TF2_WEIGHTS_NAME, has_file_kwargs): File "D:\sd-webui-aki-v4\py310\lib\site-packages\transformers\utils\hub.py", line 610, in has_file r = requests.head(url, headers=headers, allow_redirects=False, proxies=proxies, timeout=10) File "D:\sd-webui-aki-v4\py310\lib\site-packages\requests\api.py", line 100, in head return request("head", url, kwargs) File "D:\sd-webui-aki-v4\py310\lib\site-packages\requests\api.py", line 59, in request return session.request(method=method, url=url, kwargs) File "D:\sd-webui-aki-v4\py310\lib\site-packages\requests\sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "D:\sd-webui-aki-v4\py310\lib\site-packages\requests\sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "D:\sd-webui-aki-v4\py310\lib\site-packages\requests\adapters.py", line 507, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /openai/clip-vit-large-patch14/resolve/main/tf_model.h5 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002125AD0DE40>, 'Connection to huggingface.co timed out. (connect timeout=10)')) Missing model directory, removing model: D:\sd-webui-aki-v4\models\dreambooth\test\working\text_encoder Restored system models. Duration: 00:01:12 Exception loading config: [Errno 2] No such file or directory: 'D:\sd-webui-aki-v4\models\dreambooth\test\db_config.json' 提示：Python 运行时抛出了一个异常。请检查疑难解答页面。 Traceback (most recent call last): File "D:\sd-webui-aki-v4\extensions\sd_dreambooth_extension\dreambooth\dataclasses\db_config.py", line 410, in from_file with open(config_file, 'r') as openfile: FileNotFoundError: [Errno 2] No such file or directory: 'D:\sd-webui-aki-v4\models\dreambooth\test\db_config.json' 提示：Python 运行时抛出了一个异常。请检查疑难解答页面。 Can't load config! Traceback (most recent call last): File "D:\sd-webui-aki-v4\py310\lib\site-packages\gradio\routes.py", line 422, in run_predict output = await app.get_blocks().process_api( File "D:\sd-webui-aki-v4\py310\lib\site-packages\gradio\blocks.py", line 1326, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "D:\sd-webui-aki-v4\py310\lib\site-packages\gradio\blocks.py", line 1229, in postprocess_data self.validate_outputs(fn_index, predictions) # type: ignore File "D:\sd-webui-aki-v4\py310\lib\site-packages\gradio\blocks.py", line 1204, in validate_outputs raise ValueError( ValueError: An event handler (load_model_params) didn't receive enough output values (needed: 10, received: 7). 提示：Python 运行时抛出了一个异常。请检查疑难解答页面。 Wanted outputs: [html, html, html, html, html, html, html, dropdown, dropdown, html] Received outputs: ["", "", "", "", "", {'visible': True, 'choices': [], 'value': '', 'type': 'generic_update'}, "Error loading model params: 'test'."]

--------------------------------------------------------------
I also have this problem. Is this the reason for my network
-------------------------------------------------------------