search

lllyasviel / stable-diffusion-webui-forge

GNU Affero General Public License v3.0
7.06k stars 689 forks source link

[Bug]: My vae doesn't seem to load when I'm using the XL model. #678

Open dernuiJK opened 4 months ago

dernuiJK commented 4 months ago

Checklist

What happened?

My vae doesn't seem to load when I'm using the XL model. The picture that comes out is the picture that will only appear if VAE is not used, but the VAE I used is shown below the generated picture.

Steps to reproduce the problem

正常的操作步骤

What should have happened?

生成正常的图片

What browsers do you use to access the UI ?

Microsoft Edge

Sysinfo

sysinfo-2024-04-19-15-49.json

Console logs

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: f0.0.17v1.8.0rc-latest-276-g29be1da7
Commit hash: 29be1da7cf2b5dccfc70fbdd33eb35c56a31ffb7
Path A:\ComfyUI-ainewsto\ComfyUI\models\Stable-diffusion does not exist. Skip setting --ckpt-dir
Path A:\ComfyUI-ainewsto\ComfyUI\models\VAE does not exist. Skip setting --vae-dir
Path A:\ComfyUI-ainewsto\ComfyUI\models\hypernetworks does not exist. Skip setting --hypernetwork-dir
Path A:\ComfyUI-ainewsto\ComfyUI\embeddings does not exist. Skip setting --embeddings-dir
Path A:\ComfyUI-ainewsto\ComfyUI\models\lora does not exist. Skip setting --lora-dir
Path A:\ComfyUI-ainewsto\ComfyUI\models\ControlNet does not exist. Skip setting --controlnet-dir
Path A:\ComfyUI-ainewsto\ComfyUI\extensions\sd-webui-controlnet\annotator\downloads does not exist. Skip setting --controlnet-preprocessor-models-dir
Launching Web UI with arguments: --forge-ref-a1111-home A:\ComfyUI-ainewsto\ComfyUI
Total VRAM 8192 MB, total RAM 32670 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3070 Laptop GPU : native
Hint: your device supports --pin-shared-memory for potential speed improvements.
Hint: your device supports --cuda-malloc for potential speed improvements.
Hint: your device supports --cuda-stream for potential speed improvements.
VAE dtype: torch.bfloat16
CUDA Stream Activated:  False
Using pytorch cross attention
ControlNet preprocessor location: D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\ControlNetPreprocessor
Checkpoint realisticVisionV51_v51VAE.safetensors not found; loading fallback animeIllustDiffusion_v06.safetensors
Calculating sha256 for D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\Stable-diffusion\animeIllustDiffusion_v06.safetensors: 2024-04-19 23:41:07,784 - ControlNet - INFO - ControlNet UI callback registered.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 18.1s (prepare environment: 4.0s, import torch: 5.3s, import gradio: 1.4s, setup paths: 1.2s, initialize shared: 0.2s, other imports: 0.7s, setup gfpgan: 0.2s, list SD models: 0.1s, load scripts: 3.8s, create ui: 0.7s, gradio launch: 0.4s).
3137f0f2c6804a0106a8bbe320c7b4d0fa41439c1adc91f1dc9eab8a394d9832
Loading weights [3137f0f2c6] from D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\Stable-diffusion\animeIllustDiffusion_v06.safetensors
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale'}
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7117.99609375
[Memory Management] Model Memory (MB) =  2144.3546981811523
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3949.6413955688477
Moving model(s) has taken 0.75 seconds
Model loaded in 11.6s (calculate hash: 6.4s, forge instantiate config: 0.9s, forge load real models: 3.1s, calculate empty prompt: 1.0s).
Calculating sha256 for D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\Stable-diffusion\kohakuXLEpsilon_rev1.safetensors: 516047db97160d4ff96232838d7696d4a5b659e6a2224018e199acd17ef99113
Loading weights [516047db97] from D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\Stable-diffusion\kohakuXLEpsilon_rev1.safetensors
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_g.logit_scale', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids'}
Loading VAE weights found near the checkpoint: D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\VAE\kohakuXLEpsilon_rev1.safetensors
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7077.265625
[Memory Management] Model Memory (MB) =  2144.3546981811523
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3908.9109268188477
Moving model(s) has taken 0.73 seconds
Model loaded in 63.6s (unload existing model: 0.9s, calculate hash: 5.3s, forge instantiate config: 1.0s, forge load real models: 54.9s, load VAE: 0.5s, calculate empty prompt: 0.9s).
Calculating sha256 for D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\VAE\kohakuXLEpsilon_rev1.safetensors: 551eac7037ce58de1ef4447f16d48664c1e67f0e27af50a06b1a6d458b571e0c
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7075.068359375
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  1153.9818649291992
Moving model(s) has taken 2.08 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:25<00:00,  1.26s/it]
To load target model AutoencoderKL█████████████████████████████████████████████████████| 20/20 [00:22<00:00,  1.22s/it]
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7036.669921875
[Memory Management] Model Memory (MB) =  159.55708122253418
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  5853.112840652466
Moving model(s) has taken 1.05 seconds
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:26<00:00,  1.30s/it]
To load target model SDXL██████████████████████████████████████████████████████████████| 20/20 [00:26<00:00,  1.22s/it]
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7040.8017578125
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  1119.7152633666992
Moving model(s) has taken 1.68 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:24<00:00,  1.22s/it]
To load target model AutoencoderKL█████████████████████████████████████████████████████| 20/20 [00:23<00:00,  1.23s/it]
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7028.4033203125
[Memory Management] Model Memory (MB) =  159.55708122253418
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  5844.846239089966
Moving model(s) has taken 1.21 seconds
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:26<00:00,  1.31s/it]
Calculating sha256 for D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\Stable-diffusion\bluePencilXL_v200.safetensors: d59627b7495b0581ded2b0488ac98f9deb86b52d43e9d902914f0f9dcffdb16c
Loading weights [d59627b749] from D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\Stable-diffusion\bluePencilXL_v200.safetensors
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale'}
Loading VAE weights specified in settings: D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\VAE\kohakuXLEpsilon_rev1.safetensors
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7041.67822265625
[Memory Management] Model Memory (MB) =  2144.3546981811523
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  3873.3235244750977
Moving model(s) has taken 0.74 seconds
Model loaded in 28.2s (unload existing model: 1.4s, calculate hash: 4.9s, load weights from disk: 0.5s, forge instantiate config: 1.0s, forge load real models: 19.0s, load VAE: 0.5s, calculate empty prompt: 0.8s).
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7041.068359375
[Memory Management] Model Memory (MB) =  4897.086494445801
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  1119.9818649291992
Moving model(s) has taken 1.84 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:24<00:00,  1.23s/it]
To load target model AutoencoderKL█████████████████████████████████████████████████████| 20/20 [00:23<00:00,  1.21s/it]
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  7028.669921875
[Memory Management] Model Memory (MB) =  159.55708122253418
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  5845.112840652466
Moving model(s) has taken 1.05 seconds
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:26<00:00,  1.32s/it]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:26<00:00,  1.21s/it]

Additional information

No response

Niburan commented 4 months ago

Looks like you're using the same model as checkpoint as well as VAE

Loading weights [516047db97] from D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\Stable-diffusion\kohakuXLEpsilon_rev1.safetensors

Loading VAE weights found near the checkpoint: D:\webui_forge_cu121_torch21\forgeUI-N_2\webui\models\VAE\kohakuXLEpsilon_rev1.safetensors