search

vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.56k stars 409 forks source link

[Issue]: cannot load the LCM or SSD-1B or SDXL models with diffusers #2481

Closed iwoolf closed 11 months ago

iwoolf commented 11 months ago

Issue Description

On the latest version of sdnext, I have diffusers chosen as the execution backend. I cannot load LCM or SSD-1B or SDXL models due to diffusers errors, but Diffusion 1.5 and 2.1 models load OK. I have removed and reinstalled diffusers. I'm running a conda environment.

Version Platform Description

Ubuntu 22.04 LTS, NVIDIA 3060 12Gb arch=x86_64 cpu=x86_64 system=Linux release=5.15.0-88-generic python=3.10.12 Chromium Version 119.0.6045.105 (Official Build) Built on Ubuntu , running on elementary 7.1 Browser session: client=127.0.0.1 agent=Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/119.0.0.0 Safari/537.36

Relevant log output

./webui.sh
Create and activate python venv
Launching launch.py...
11:42:05-712203 INFO     Starting SD.Next                                       
11:42:05-718147 INFO     Python 3.10.12 on Linux                                
11:42:05-754096 INFO     Version: app=sd.next updated=2023-11-09 hash=baed8728  
                         url=https://github.com/vladmandic/automatic.git/tree/ma
                         ster                                                   
11:42:06-279526 INFO     Platform: arch=x86_64 cpu=x86_64 system=Linux          
                         release=5.15.0-88-generic python=3.10.12               
11:42:06-282565 INFO     nVidia CUDA toolkit detected: nvidia-smi present       
11:42:06-295456 WARNING  Modified files: ['=0.0.2', '=1.9.3', '=2.0.0',         
                         '=2.1.9', '=3.0.0', '=4.3.1', '=8.1.3']                
11:42:06-373237 INFO     Extensions: disabled=[]                                
11:42:06-374621 INFO     Extensions: enabled=['a1111-sd-webui-lycoris',         
                         'sd-extension-chainner', 'ScuNET',                     
                         'multidiffusion-upscaler-for-automatic1111',           
                         'stable-diffusion-webui-images-browser', 'LDSR',       
                         'clip-interrogator-ext', 'sd-extension-system-info',   
                         'Lora', 'SwinIR', 'sd-dynamic-thresholding',           
                         'stable-diffusion-webui-rembg',                        
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet']     
                         extensions-builtin                                     
11:42:06-377436 INFO     Extensions: enabled=[] extensions                      
11:42:06-379083 INFO     Startup: quick launch                                  
11:42:06-379972 INFO     Verifying requirements                                 
11:42:06-424459 INFO     Verifying packages                                     
11:42:06-428012 INFO     Extensions: disabled=[]                                
11:42:06-429079 INFO     Extensions: enabled=['a1111-sd-webui-lycoris',         
                         'sd-extension-chainner', 'ScuNET',                     
                         'multidiffusion-upscaler-for-automatic1111',           
                         'stable-diffusion-webui-images-browser', 'LDSR',       
                         'clip-interrogator-ext', 'sd-extension-system-info',   
                         'Lora', 'SwinIR', 'sd-dynamic-thresholding',           
                         'stable-diffusion-webui-rembg',                        
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet']     
                         extensions-builtin                                     
11:42:06-431257 INFO     Extensions: enabled=[] extensions                      
11:42:06-435425 INFO     Extension preload: {'extensions-builtin': 0.0,         
                         'extensions': 0.0}                                     
11:42:06-436984 INFO     Command line args: []                                  
11:42:19-920580 INFO     Load packages: torch=2.0.1+cu118 diffusers=0.23.0      
                         gradio=3.43.2                                          
11:42:21-980151 INFO     Engine: backend=Backend.DIFFUSERS compute=cuda         
                         mode=no_grad device=cuda                               
                         cross-optimization="Scaled-Dot-Product"                
11:42:22-076490 INFO     Device: device=NVIDIA GeForce RTX 3060 n=1 arch=sm_90  
                         cap=(8, 6) cuda=11.8 cudnn=8700 driver=535.129.03      
11:42:23-457551 INFO     Available VAEs: path="models/VAE" items=0              
11:42:23-459066 INFO     Disabling uncompatible extensions:                     
                         backend=Backend.DIFFUSERS ['sd-webui-controlnet',      
                         'multidiffusion-upscaler-for-automatic1111',           
                         'a1111-sd-webui-lycoris']                              
11:42:23-524645 INFO     Available models: path="/media/iwoolf/tenT/SDModels"   
                         items=68 time=0.06                                     
11:42:26-266121 INFO     Extension:                                             
                         script='extensions-builtin/sd-dynamic-thresholding/scri
                         pts/dynamic_thresholding.py' ======                    
11:42:26-292646 INFO     Extension:                                             
                         script='extensions-builtin/sd-dynamic-thresholding/scri
                         pts/dynamic_thresholding.py' Error! UniPC sampler      
                         support failed to load! Is your WebUI up to date?      
11:42:26-294104 INFO     Extension:                                             
                         script='extensions-builtin/sd-dynamic-thresholding/scri
                         pts/dynamic_thresholding.py' (Error: module            
                         'modules.models.diffusion.uni_pc' has no attribute     
                         'sampler')                                             
11:42:26-295615 INFO     Extension:                                             
                         script='extensions-builtin/sd-dynamic-thresholding/scri
                         pts/dynamic_thresholding.py' ======                    
11:42:30-660421 INFO     Extensions time: 6.96 { clip-interrogator-ext=1.47     
                         Lora=0.91 sd-dynamic-thresholding=0.20                 
                         sd-extension-chainner=0.20                             
                         sd-extension-system-info=0.19                          
                         sd-webui-agent-scheduler=1.97                          
                         stable-diffusion-webui-images-browser=0.34             
                         stable-diffusion-webui-rembg=1.66 }                    
11:42:31-388462 INFO     Load UI theme: name="black-teal" style=Auto            
                         base=style.css                                         
11:42:35-758164 INFO     Local URL: http://127.0.0.1:7860/                      
11:42:35-759650 INFO     Initializing middleware                                
11:42:36-764252 INFO     [AgentScheduler] Task queue is empty                   
11:42:36-765910 INFO     [AgentScheduler] Registering APIs                      
11:42:37-139425 INFO     Select: model="LCM_Dreamshaper_v7_4k [84feab3a32]"     
Loading weights: /media/iwoolf/tenT/SDModels/LCM_Dreamshaper_v7_4k.safetensors  
11:42:37-434439 INFO     Setting Torch parameters: device=cuda                  
                         dtype=torch.float16 vae=torch.float16                  
                         unet=torch.float16 context=no_grad fp16=True bf16=False
11:42:37-436298 INFO     Autodetect: model="Latent Consistency Model"           
                         class=LatentConsistencyModelPipeline                   
                         file="/media/iwoolf/tenT/SDModels/LCM_Dreamshaper_v7_4k
                         .safetensors" size=3279MB                              
11:42:37-443320 ERROR    Diffusers failed loading:                              
                         model=/media/iwoolf/tenT/SDModels/LCM_Dreamshaper_v7_4k
                         .safetensors pipeline=Autodetect/NoneType Unhandled    
                         pipeline class: LatentConsistencyModelPipeline         
11:42:37-471019 INFO     Startup time: 30.96 { torch=12.23 gradio=1.16          
                         diffusers=0.05 libraries=3.51 extensions=6.96          
                         models=0.07 face-restore=0.18 upscalers=0.39           
                         ui-extra-networks=0.88 ui-txt2img=0.13 ui-img2img=0.25 
                         ui-train=0.06 ui-models=0.06 ui-settings=0.53          
                         ui-extensions=1.76 ui-defaults=0.25 launch=0.68        
                         api=0.28 app-started=1.09 checkpoint=0.31 }            
11:42:59-369092 INFO     MOTD: N/A                                              
11:43:15-263246 INFO     Browser session: client=127.0.0.1 agent=Mozilla/5.0    
                         (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like    
                         Gecko) Chrome/119.0.0.0 Safari/537.36                  
11:43:33-697495 INFO     Select: model="Realistic_Vision_V2.0"                  
Loading weights: /media/iwoolf/tenT/SDModels/Realistic_Vision_V2.0.safetensors  
11:43:42-416208 INFO     Setting Torch parameters: device=cuda                  
                         dtype=torch.float16 vae=torch.float16                  
                         unet=torch.float16 context=no_grad fp16=True bf16=False
11:43:42-418817 INFO     Autodetect: model="Stable Diffusion"                   
                         class=StableDiffusionPipeline                          
                         file="/media/iwoolf/tenT/SDModels/Realistic_Vision_V2.0
                         .safetensors" size=3673MB                              
Calculating hash: /media/iwoolf/tenT/SDModels/Realistic_Vision_V2.0.safetensors 
11:45:22-451401 INFO     Load embeddings: loaded=8 skipped=3 time=2.10          
11:45:27-121981 INFO     Load model: time=108.86 { load=108.86 } native=512     
                         {'ram': {'used': 2.36, 'total': 15.52}, 'gpu': {'used':
                         3.16, 'total': 11.76}, 'retries': 0, 'oom': 0}         
11:49:30-666076 INFO     Select: model="realisticVisionV51_v51VAE"              
Loading weights: /media/iwoolf/tenT/SDModels/realisticVisionV51_v51VAE.safeten… 
11:49:38-431331 INFO     Setting Torch parameters: device=cuda                  
                         dtype=torch.float16 vae=torch.float16                  
                         unet=torch.float16 context=no_grad fp16=True bf16=False
11:49:38-435413 INFO     Autodetect: model="Stable Diffusion"                   
                         class=StableDiffusionPipeline                          
                         file="/media/iwoolf/tenT/SDModels/realisticVisionV51_v5
                         1VAE.safetensors" size=2034MB                          
Calculating hash: /media/iwoolf/tenT/SDModels/realisticVisionV51_v51VAE.safete… 
11:50:44-627756 INFO     Load embeddings: loaded=8 skipped=3 time=2.11          
11:50:49-242361 INFO     Load model: time=70.71 { load=70.67 } native=512       
                         {'ram': {'used': 2.88, 'total': 15.52}, 'gpu': {'used':
                         3.19, 'total': 11.76}, 'retries': 0, 'oom': 0}         
11:54:37-637422 INFO     Select: model="SSD-1B"                                 
Loading weights: /media/iwoolf/tenT/SDModels/SSD-1B.safetensors ━━━━ 0.0… -:--:…
                                                                     GB         
11:54:52-087640 INFO     Setting Torch parameters: device=cuda                  
                         dtype=torch.float16 vae=torch.float16                  
                         unet=torch.float16 context=no_grad fp16=True bf16=False
11:54:52-111391 INFO     Autodetect: model="Stable Diffusion"                   
                         class=StableDiffusionPipeline                          
                         file="/media/iwoolf/tenT/SDModels/SSD-1B.safetensors"  
                         size=4259MB                                            
11:54:54-632685 ERROR    Diffusers failed loading:                              
                         model=/media/iwoolf/tenT/SDModels/SSD-1B.safetensors   
                         pipeline=Autodetect/NoneType 1                         
11:55:07-013504 INFO     Select: model="LCM_Dreamshaper_v7_4k [84feab3a32]"     
Loading weights: /media/iwoolf/tenT/SDModels/LCM_Dreamshaper_v7_4k.safetensors  
11:55:10-496561 INFO     Setting Torch parameters: device=cuda                  
                         dtype=torch.float16 vae=torch.float16                  
                         unet=torch.float16 context=no_grad fp16=True bf16=False
11:55:10-498659 INFO     Autodetect: model="Latent Consistency Model"           
                         class=LatentConsistencyModelPipeline                   
                         file="/media/iwoolf/tenT/SDModels/LCM_Dreamshaper_v7_4k
                         .safetensors" size=3279MB                              
11:55:10-500081 ERROR    Diffusers failed loading:                              
                         model=/media/iwoolf/tenT/SDModels/LCM_Dreamshaper_v7_4k
                         .safetensors pipeline=Autodetect/NoneType Unhandled    
                         pipeline class: LatentConsistencyModelPipeline         
11:55:32-967789 INFO     Select: model="sd_xl_base_1.0"                         
Loading weights: /media/iwoolf/tenT/SDModels/sd_xl_base_1.0.safetensors ━━ 0… -…
                                                                           GB   
11:55:46-749433 INFO     Setting Torch parameters: device=cuda                  
                         dtype=torch.float16 vae=torch.float16                  
                         unet=torch.float16 context=no_grad fp16=True bf16=False
11:55:46-751720 INFO     Autodetect: model="Stable Diffusion XL"                
                         class=StableDiffusionXLPipeline                        
                         file="/media/iwoolf/tenT/SDModels/sd_xl_base_1.0.safete
                         nsors" size=6617MB

Backend

Original

Model

SD-XL

Acknowledgements

vladmandic commented 11 months ago

notes clearly stated that ANY models other than standard SD15/SDXL must be loaded using huggingface format, not safetensors. safetensors simply does not have sufficient information - it doesn't even tell us what model type it is.

and this is fully documented.

so go to networks -> reference -> select model and it will be downloaded and loaded.

and do not do things like remove/install diffusers yourself, app takes care of its requirements.