lllyasviel / sd-forge-layerdiffuse

[WIP] Layer Diffusion for WebUI (via Forge)
Apache License 2.0
3.89k stars 334 forks source link

(SD1.5) From Background to Foreground (need batch size 2) #125

Open G7b9 opened 3 months ago

G7b9 commented 3 months ago

(AttributeError: 'Linear' object has no attribute 'in_features')Errors occur when I use the realisticVisionV51_v51VAE.safetensors。

`Python 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0] Version: f2.0.1v1.10.1-previous-399-g852e8856 Commit hash: 852e88564c871ef640c12ac053e52234958653d8 Launching Web UI with arguments: --skip-install --api --xformers --ckpt-dir ../stable-diffusion-webui/models/Stable-diffusion --vae-dir ../stable-diffusion-webui/models/VAE --hypernetwork-dir ../stable-diffusion-webui/models/hypernetworks --embeddings-dir ../stable-diffusion-webui/embeddings --lora-dir ../stable-diffusion-webui/models/Lora --controlnet-dir ../stable-diffusion-webui/models/ControlNet --controlnet-preprocessor-models-dir ../stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads Total VRAM 7863 MB, total RAM 64188 MB pytorch version: 2.3.0+cu121 xformers version: 0.0.26.post1 Set vram state to: NORMAL_VRAM Device: cuda:0 NVIDIA GeForce RTX 3060 : native Hint: your device supports --cuda-malloc for potential speed improvements. VAE dtype preferences: [torch.bfloat16, torch.float32] -> torch.bfloat16 CUDA Using Stream: False Using xformers cross attention Using xformers attention for VAE

You are running torch 2.3.0+cu121. The program is tested to work with torch 2.3.1. To reinstall the desired version, run with commandline flag --reinstall-torch. Beware that this will cause a lot of large files to be downloaded, as well as there are reports of issues with training tab on the latest version.

Use --skip-version-check commandline argument to disable this check.

================================================================================= You are running xformers 0.0.26.post1. The program is tested to work with xformers 0.0.27. To reinstall the desired version, run with commandline flag --reinstall-xformers.

Use --skip-version-check commandline argument to disable this check.

ControlNet preprocessor location: ../stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads 2024-08-22 15:45:23,246 - ControlNet - INFO - ControlNet UI callback registered. Model selected: {'checkpoint_info': {'filename': '/mnt/data/stable-diffusion-webui-forge/../stable-diffusion-webui/models/Stable-diffusion/realisticVisionV51_v51VAE.safetensors', 'hash': 'a0f13c83'}, 'additional_modules': [], 'unet_storage_dtype': None} Using online LoRAs in FP16: False Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Startup time: 18.9s (prepare environment: 1.9s, import torch: 7.5s, initialize shared: 0.3s, other imports: 0.4s, list SD models: 0.5s, load scripts: 1.5s, create ui: 2.8s, gradio launch: 1.8s, add APIs: 2.1s). Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False} [GPU Setting] You will use 86.98% GPU memory (6839.00 MB) to load weights, and use 13.02% GPU memory (1024.00 MB) to do matrix computation. Loading Model: {'checkpoint_info': {'filename': '/mnt/data/stable-diffusion-webui-forge/../stable-diffusion-webui/models/Stable-diffusion/realisticVisionV51_v51VAE.safetensors', 'hash': 'a0f13c83'}, 'additional_modules': [], 'unet_storage_dtype': None} [Unload] Trying to free all memory for cuda:0 with 0 models keep loaded ... StateDict Keys: {'unet': 686, 'vae': 248, 'text_encoder': 197, 'ignore': 0} Working with z of shape (1, 4, 32, 32) = 4096 dimensions. K-Model Created: {'storage_dtype': torch.float16, 'computation_dtype': torch.float16} Model loaded in 0.9s (unload existing model: 0.3s, forge model load: 0.6s). To load target model JointTextEncoder Begin to load 1 model [Unload] Trying to free 1329.14 MB for cuda:0 with 0 models keep loaded ... [Memory Management] Current Free GPU Memory: 6656.83 MB [Memory Management] Required Model Memory: 234.72 MB [Memory Management] Required Inference Memory: 1024.00 MB [Memory Management] Estimated Remaining GPU Memory: 5398.11 MB Moving model(s) has taken 0.07 seconds [Unload] Trying to free 1024.00 MB for cuda:0 with 1 models keep loaded ... [Unload] Current free memory is 6311.67 MB ... [LayerDiffuse] LayerMethod.BG_TO_FG_SD15 Error running process_before_every_sampling: /mnt/data/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/scripts/forge_layerdiffusion.py Traceback (most recent call last): File "/mnt/data/stable-diffusion-webui-forge/modules/scripts.py", line 892, in process_before_every_sampling script.process_before_every_sampling(p, script_args, kwargs) File "/mnt/data/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/scripts/forge_layerdiffusion.py", line 236, in process_before_every_sampling patcher = AttentionSharingPatcher(unet, frames=2, use_control=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/lib_layerdiffusion/attention_sharing.py", line 207, in init u = AttentionSharingUnit(attn_module, frames=frames, use_control=use_control, rank=rank) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/lib_layerdiffusion/attention_sharing.py", line 35, in init q_in_channels, q_out_channels = module.to_q.in_features, module.to_q.out_features ^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/stable-diffusion-webui/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1709, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'Linear' object has no attribute 'in_features'


To load target model KModel Begin to load 1 model [Unload] Trying to free 3155.23 MB for cuda:0 with 0 models keep loaded ... [Unload] Current free memory is 6311.29 MB ... [Memory Management] Current Free GPU Memory: 6311.29 MB [Memory Management] Required Model Memory: 1639.41 MB [Memory Management] Required Inference Memory: 1024.00 MB [Memory Management] Estimated Remaining GPU Memory: 3647.88 MB Moving model(s) has taken 0.47 seconds 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 3.03it/s] To load target model IntegratedAutoencoderKL███████████████████████████████████████████████████████| 20/20 [00:06<00:00, 3.10it/s] Begin to load 1 model [Unload] Trying to free 1568.67 MB for cuda:0 with 0 models keep loaded ... [Unload] Current free memory is 4647.28 MB ... [Memory Management] Current Free GPU Memory: 4647.28 MB [Memory Management] Required Model Memory: 159.56 MB [Memory Management] Required Inference Memory: 1024.00 MB [Memory Management] Estimated Remaining GPU Memory: 3463.73 MB Moving model(s) has taken 0.04 seconds To load target model UNet1024 Begin to load 1 model [Unload] Trying to free 1282.13 MB for cuda:0 with 0 models keep loaded ... [Unload] Current free memory is 4467.43 MB ... [Memory Management] Current Free GPU Memory: 4467.43 MB [Memory Management] Required Model Memory: 198.56 MB [Memory Management] Required Inference Memory: 1024.00 MB [Memory Management] Estimated Remaining GPU Memory: 3244.87 MB Moving model(s) has taken 0.07 seconds 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00, 6.96it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████████████████| 20/20 [00:09<00:00, 2.09it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████████████████| 20/20 [00:09<00:00, 3.10it/s]` image image