vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.53k stars 406 forks source link

[Issue]: webui.bat stuck at / after loading UI theme #2341

Closed nCoderGit closed 11 months ago

nCoderGit commented 11 months ago

Issue Description

Something must have broken right after I pulled the "fixes for zero-day issues" commit (231ca1a). Webui.bat suddenly gets stuck at 19:51:04-625209 INFO Loading UI theme: name=black-teal style=Auto base=style.css with ~25% CPU usage for python.exe everytime I try to start SD.

This also prevents the server from starting, so SD is unusable for me right now.

(Deleting the venv folder didn't help.)

Version Platform Description

Win10, 48GB RAM, RTX 3060TI 8GB

Relevant log output

Using VENV: C:\StableDiffusion\SDNext\venv
19:49:57-548440 INFO     Starting SD.Next
19:49:57-554442 INFO     Python 3.10.6 on Windows
19:49:57-735331 INFO     Version: app=sd.next updated=2023-10-18 hash=76d262f1 url=https://github.com/vladmandic/automatic/tree/master
19:49:58-177465 INFO     Platform: arch=AMD64 cpu=Intel64 Family 6 Model 94 Stepping 3, GenuineIntel system=Windows
                         release=Windows-10-10.0.19045-SP0 python=3.10.6
19:49:58-215706 INFO     nVidia CUDA toolkit detected: nvidia-smi present
19:49:58-370015 INFO     Verifying requirements
19:49:58-390072 INFO     Verifying packages
19:49:58-394033 INFO     Verifying repositories
19:50:02-488266 INFO     Verifying submodules
19:50:46-866711 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'ScuNET',
                         'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'sd-canvas-editor',
                         'sdweb-merge-board', 'stable-diffusion-NPW', 'stable-diffusion-webui-embedding-merge',
                         'stable-diffusion-webui-state']
19:50:46-873998 INFO     Verifying packages
19:50:46-875983 INFO     Updating Wiki
19:50:47-701209 INFO     Extension preload: {'extensions-builtin': 0.01, 'extensions': 0.0}
19:50:47-704201 INFO     Command line args: ['--upgrade'] upgrade=True
19:50:57-416236 INFO     Engine: backend=Backend.ORIGINAL compute=cuda mode=no_grad device=cuda cross-optimization="Sub-quadratic"
19:50:57-490366 INFO     Device: device=NVIDIA GeForce RTX 3060 Ti n=1 arch=sm_90 cap=(8, 6) cuda=12.1 cudnn=8801 driver=536.23
19:50:58-335760 INFO     Available VAEs: ..\_SD_Models\VAE items=3
19:50:58-709735 INFO     Available models: ..\_SD_Models\Checkpoints items=105 time=0.37s
19:51:02-368495 INFO     Extension: script='extensions-builtin\sd-webui-controlnet\scripts\controlnet.py' Warning: ControlNet failed to
                         load SGM - will use LDM instead.
19:51:02-373518 INFO     Extension: script='extensions-builtin\sd-webui-controlnet\scripts\controlnet.py' ControlNet preprocessor location:
                         C:\StableDiffusion\SDNext\extensions-builtin\sd-webui-controlnet\annotator\downloads
19:51:02-390065 INFO     Extension: script='extensions-builtin\sd-webui-controlnet\scripts\hook.py' Warning: ControlNet failed to load SGM
                         - will use LDM instead.
19:51:04-304307 INFO     Extensions time: 5.27s { a1111-sd-webui-lycoris=1.58s clip-interrogator-ext=0.08s Lora=0.34s
                         sd-extension-chainner=0.12s sd-extension-system-info=0.06s sd-webui-agent-scheduler=0.54s
                         sd-webui-controlnet=0.62s stable-diffusion-webui-images-browser=0.15s stable-diffusion-webui-rembg=1.18s
                         sd-canvas-editor=0.12s sdweb-merge-board=0.08s stable-diffusion-webui-embedding-merge=0.22s
                         stable-diffusion-webui-state=0.14s }
19:51:04-625209 INFO     Loading UI theme: name=black-teal style=Auto base=style.css

Backend

Original

Model

SD 1.5

Acknowledgements

EKKIV0K commented 11 months ago

Same here @vladmandic erf

vladmandic commented 11 months ago

Please run with webui --debug --safe and upload log here.

EKKIV0K commented 11 months ago

@vladmandic i'am really disapointed it seems to work with the debug args but not on normal mode :O

EKKIV0K commented 11 months ago

erf

vladmandic commented 11 months ago

This is not a clean install, it's an upgrade, right?

EKKIV0K commented 11 months ago

@vladmandic it's an updated version via Git pull command :O, but the base is a clean version

nCoderGit commented 11 months ago

--debug --safe does work, --debug (normal mode) gets stuck at (or after?) loading the theme and before checking the extra networks.

I'll attach both the working and the broken log down below.

Yup, it's an upgrade from 379fe1f and I suspect it was the "fixes for zero-day issues" commit that broke it since I had it running just a few minutes before I pulled that one.

Broken (webui.bat --debug):

Using VENV: C:\StableDiffusion\SDNext\venv
22:47:14-873046 DEBUG    Logger: file=C:\StableDiffusion\SDNext\sdnext.log level=10 size=0 mode=create
22:47:14-883011 INFO     Starting SD.Next
22:47:14-889990 INFO     Python 3.10.6 on Windows
22:47:15-074482 INFO     Version: app=sd.next updated=2023-10-18 hash=76d262f1 url=https://github.com/vladmandic/automatic/tree/master
22:47:15-535805 INFO     Latest published version: 66f75b6a1114d07e40e27a09cd2f14f78e16eba3 2023-10-18T20:00:23Z
22:47:15-572234 INFO     Platform: arch=AMD64 cpu=Intel64 Family 6 Model 94 Stepping 3, GenuineIntel system=Windows
                         release=Windows-10-10.0.19045-SP0 python=3.10.6
22:47:15-586700 DEBUG    Setting environment tuning
22:47:15-592706 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False openvino=False
22:47:15-602864 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True openvino=True
22:47:15-609697 INFO     nVidia CUDA toolkit detected: nvidia-smi present
22:47:15-773102 DEBUG    Repository update time: Wed Oct 18 19:25:12 2023
22:47:15-777553 INFO     Verifying requirements
22:47:15-795534 INFO     Verifying packages
22:47:15-800394 INFO     Verifying repositories
22:47:15-889913 DEBUG    Submodule: C:\StableDiffusion\SDNext\repositories\stable-diffusion-stability-ai / main
22:47:16-734054 DEBUG    Submodule: C:\StableDiffusion\SDNext\repositories\taming-transformers / master
22:47:18-764993 DEBUG    Submodule: C:\StableDiffusion\SDNext\repositories\BLIP / main
22:47:19-471154 INFO     Verifying submodules
22:47:29-104116 DEBUG    Submodule: extensions-builtin/clip-interrogator-ext / main
22:47:29-194060 DEBUG    Submodule: extensions-builtin/sd-extension-chainner / main
22:47:29-286968 DEBUG    Submodule: extensions-builtin/sd-extension-system-info / main
22:47:29-375897 DEBUG    Submodule: extensions-builtin/sd-webui-agent-scheduler / main
22:47:29-467437 DEBUG    Submodule: extensions-builtin/sd-webui-controlnet / main
22:47:29-566313 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-images-browser / main
22:47:29-652648 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-rembg / master
22:47:29-739846 DEBUG    Submodule: modules/lora / main
22:47:29-830218 DEBUG    Submodule: wiki / master
22:47:30-058501 DEBUG    Installed packages: 214
22:47:30-061509 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'ScuNET',
                         'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
22:47:30-244044 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\clip-interrogator-ext\install.py
22:47:40-003314 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\sd-extension-system-info\install.py
22:47:40-626123 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\sd-webui-agent-scheduler\install.py
22:47:41-255250 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\sd-webui-controlnet\install.py
22:47:41-987531 DEBUG    Running extension installer:
                         C:\StableDiffusion\SDNext\extensions-builtin\stable-diffusion-webui-images-browser\install.py
22:47:42-696532 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\stable-diffusion-webui-rembg\install.py
22:47:43-568954 DEBUG    Extensions all: ['sd-canvas-editor', 'sdweb-merge-board', 'stable-diffusion-NPW',
                         'stable-diffusion-webui-embedding-merge', 'stable-diffusion-webui-state']
22:47:44-526674 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'ScuNET',
                         'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'sd-canvas-editor',
                         'sdweb-merge-board', 'stable-diffusion-NPW', 'stable-diffusion-webui-embedding-merge',
                         'stable-diffusion-webui-state']
22:47:44-534659 INFO     Verifying packages
22:47:44-537651 DEBUG    Setup complete without errors: 1697662065
22:47:44-549620 INFO     Extension preload: {'extensions-builtin': 0.01, 'extensions': 0.0}
22:47:44-553703 DEBUG    Starting module: <module 'webui' from 'C:\\StableDiffusion\\SDNext\\webui.py'>
22:47:44-556695 INFO     Command line args: ['--debug'] debug=True
22:47:53-950141 DEBUG    Loaded packages: torch=2.1.0+cu121 diffusers=0.21.4 gradio=3.43.2
22:47:54-651056 DEBUG    Reading: config.json len=57
22:47:54-654020 DEBUG    Unknown settings: ['realesrgan_enabled_models', 'state_txt2img', 'ldsr_steps', 'control_net_allow_script_control',
                         'upscaling_max_images_in_cache']
22:47:54-659008 INFO     Engine: backend=Backend.ORIGINAL compute=cuda mode=no_grad device=cuda cross-optimization="Sub-quadratic"
22:47:54-726028 INFO     Device: device=NVIDIA GeForce RTX 3060 Ti n=1 arch=sm_90 cap=(8, 6) cuda=12.1 cudnn=8801 driver=536.23
22:47:55-535411 DEBUG    Entering start sequence
22:47:55-540398 DEBUG    Initializing
22:47:55-548377 INFO     Available VAEs: ..\_SD_Models\VAE items=3
22:47:55-559348 DEBUG    Reading: cache.json len=2
22:47:55-761184 DEBUG    Reading: metadata.json len=1000
22:47:55-771930 INFO     Available models: ..\_SD_Models\Checkpoints items=105 time=0.22s
22:47:56-247238 DEBUG    Loading extensions
22:47:59-784368 INFO     Extension: script='extensions-builtin\sd-webui-controlnet\scripts\controlnet.py' Warning: ControlNet failed to
                         load SGM - will use LDM instead.
22:47:59-790335 INFO     Extension: script='extensions-builtin\sd-webui-controlnet\scripts\controlnet.py' ControlNet preprocessor location:
                         C:\StableDiffusion\SDNext\extensions-builtin\sd-webui-controlnet\annotator\downloads
22:47:59-806531 INFO     Extension: script='extensions-builtin\sd-webui-controlnet\scripts\hook.py' Warning: ControlNet failed to load SGM
                         - will use LDM instead.
22:48:02-201296 INFO     Extensions time: 5.95s { a1111-sd-webui-lycoris=1.47s clip-interrogator-ext=0.09s Lora=0.32s
                         sd-extension-chainner=0.13s sd-extension-system-info=0.06s sd-webui-agent-scheduler=0.68s
                         sd-webui-controlnet=0.79s stable-diffusion-webui-images-browser=0.15s stable-diffusion-webui-rembg=1.67s
                         sd-canvas-editor=0.13s sdweb-merge-board=0.09s stable-diffusion-webui-embedding-merge=0.20s
                         stable-diffusion-webui-state=0.14s }
22:48:02-376266 DEBUG    Reading: html/upscalers.json len=4
22:48:02-384325 DEBUG    Loaded upscalers: total=45 downloaded=10 user=4 ['None', 'Lanczos', 'Nearest', 'ChaiNNer', 'ESRGAN', 'LDSR',
                         'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
22:48:02-401290 DEBUG    Loaded styles: folder=models\styles items=295
22:48:02-409249 DEBUG    Creating UI
22:48:02-540720 INFO     Loading UI theme: name=black-teal style=Auto base=style.css

Working (--debug --safe):

Using VENV: C:\StableDiffusion\SDNext\venv
22:52:08-878831 DEBUG    Logger: file=C:\StableDiffusion\SDNext\sdnext.log level=10 size=0 mode=create
22:52:08-886787 INFO     Starting SD.Next
22:52:08-889779 INFO     Python 3.10.6 on Windows
22:52:09-072667 INFO     Version: app=sd.next updated=2023-10-18 hash=76d262f1 url=https://github.com/vladmandic/automatic/tree/master
22:52:09-378565 INFO     Latest published version: 66f75b6a1114d07e40e27a09cd2f14f78e16eba3 2023-10-18T20:00:23Z
22:52:09-418319 INFO     Platform: arch=AMD64 cpu=Intel64 Family 6 Model 94 Stepping 3, GenuineIntel system=Windows
                         release=Windows-10-10.0.19045-SP0 python=3.10.6
22:52:09-434085 DEBUG    Setting environment tuning
22:52:09-440068 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False openvino=False
22:52:09-447391 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True openvino=True
22:52:09-453378 INFO     nVidia CUDA toolkit detected: nvidia-smi present
22:52:09-624730 DEBUG    Repository update time: Wed Oct 18 19:25:12 2023
22:52:09-627723 INFO     Verifying requirements
22:52:09-645702 INFO     Verifying packages
22:52:09-648637 INFO     Verifying repositories
22:52:09-732433 DEBUG    Submodule: C:\StableDiffusion\SDNext\repositories\stable-diffusion-stability-ai / main
22:52:10-549913 DEBUG    Submodule: C:\StableDiffusion\SDNext\repositories\taming-transformers / master
22:52:12-683903 DEBUG    Submodule: C:\StableDiffusion\SDNext\repositories\BLIP / main
22:52:13-456386 INFO     Verifying submodules
22:52:22-976617 DEBUG    Submodule: extensions-builtin/clip-interrogator-ext / main
22:52:23-067038 DEBUG    Submodule: extensions-builtin/sd-extension-chainner / main
22:52:23-158913 DEBUG    Submodule: extensions-builtin/sd-extension-system-info / main
22:52:23-248198 DEBUG    Submodule: extensions-builtin/sd-webui-agent-scheduler / main
22:52:23-334034 DEBUG    Submodule: extensions-builtin/sd-webui-controlnet / main
22:52:23-432887 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-images-browser / main
22:52:23-526605 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-rembg / master
22:52:23-614676 DEBUG    Submodule: modules/lora / main
22:52:23-701417 DEBUG    Submodule: wiki / master
22:52:23-923097 DEBUG    Installed packages: 214
22:52:23-927001 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'ScuNET',
                         'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
22:52:24-106462 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\clip-interrogator-ext\install.py
22:52:33-915682 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\sd-extension-system-info\install.py
22:52:34-557961 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\sd-webui-agent-scheduler\install.py
22:52:35-194258 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\sd-webui-controlnet\install.py
22:52:35-834715 DEBUG    Running extension installer:
                         C:\StableDiffusion\SDNext\extensions-builtin\stable-diffusion-webui-images-browser\install.py
22:52:36-464372 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions-builtin\stable-diffusion-webui-rembg\install.py
22:52:37-282081 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'ScuNET',
                         'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
22:52:37-290205 INFO     Verifying packages
22:52:37-293197 DEBUG    Setup complete without errors: 1697662357
22:52:37-295027 INFO     Running in safe mode without user extensions
22:52:37-307912 INFO     Extension preload: {'extensions-builtin': 0.01}
22:52:37-310905 DEBUG    Starting module: <module 'webui' from 'C:\\StableDiffusion\\SDNext\\webui.py'>
22:52:37-313897 INFO     Command line args: ['--debug', '--safe'] debug=True safe=True
22:52:46-922893 DEBUG    Loaded packages: torch=2.1.0+cu121 diffusers=0.21.4 gradio=3.43.2
22:52:47-740355 DEBUG    Reading: config.json len=57
22:52:47-743323 DEBUG    Unknown settings: ['realesrgan_enabled_models', 'state_txt2img', 'ldsr_steps', 'control_net_allow_script_control',
                         'upscaling_max_images_in_cache']
22:52:47-747313 INFO     Engine: backend=Backend.ORIGINAL compute=cuda mode=no_grad device=cuda cross-optimization="Sub-quadratic"
22:52:47-813418 INFO     Device: device=NVIDIA GeForce RTX 3060 Ti n=1 arch=sm_90 cap=(8, 6) cuda=12.1 cudnn=8801 driver=536.23
22:52:48-723488 DEBUG    Entering start sequence
22:52:48-729504 DEBUG    Initializing
22:52:48-738497 INFO     Available VAEs: ..\_SD_Models\VAE items=3
22:52:48-743299 INFO     Safe mode disabling extensions: ['sd-webui-controlnet', 'a1111-sd-webui-lycoris', 'sd-webui-agent-scheduler',
                         'clip-interrogator-ext', 'stable-diffusion-webui-rembg', 'sd-extension-chainner']
22:52:48-749258 DEBUG    Reading: cache.json len=2
22:52:48-954709 DEBUG    Reading: metadata.json len=1000
22:52:48-966054 INFO     Available models: ..\_SD_Models\Checkpoints items=105 time=0.22s
22:52:49-295682 DEBUG    Loading extensions
22:52:51-104655 INFO     Extensions time: 1.81s { Lora=1.57s sd-extension-system-info=0.06s stable-diffusion-webui-images-browser=0.15s }
22:52:51-258044 DEBUG    Reading: html/upscalers.json len=4
22:52:51-265005 DEBUG    Loaded upscalers: total=30 downloaded=10 user=4 ['None', 'Lanczos', 'Nearest', 'ESRGAN', 'LDSR', 'RealESRGAN',
                         'SCUNet', 'SD', 'SwinIR']
22:52:51-280976 DEBUG    Loaded styles: folder=models\styles items=295
22:52:51-288928 DEBUG    Creating UI
22:52:51-392112 INFO     Loading UI theme: name=black-teal style=Auto base=style.css
22:52:51-859226 DEBUG    Extra networks: page='model' items=105 subdirs=1 tab=txt2img dirs=['..\\_SD_Models\\Checkpoints',
                         '..\\SD_Models\\Diffusers', 'C:\\StableDiffusion\\SDNext\\models\\Stable-diffusion'] time=0.15s
22:52:51-887359 DEBUG    Extra networks: page='style' items=295 subdirs=2 tab=txt2img dirs=['models\\styles', 'html'] time=0.02s
22:52:51-898378 DEBUG    Extra networks: page='embedding' items=33 subdirs=2 tab=txt2img dirs=['..\\_SD_Models\\TextualEmbeddings']
                         time=0.07s
22:52:51-908353 DEBUG    Extra networks: page='hypernetwork' items=0 subdirs=0 tab=txt2img dirs=['models\\hypernetworks'] time=0.0s
22:52:51-914337 DEBUG    Extra networks: page='vae' items=3 subdirs=1 tab=txt2img dirs=['..\\_SD_Models\\VAE'] time=0.01s
22:52:51-938300 DEBUG    Extra networks: page='lora' items=270 subdirs=5 tab=txt2img dirs=['C:\\StableDiffusion\\_SD_Models\\Lora',
                         'C:\\StableDiffusion\\_SD_Models\\LyCORIS'] time=0.41s
22:52:52-372512 DEBUG    Reading: ui-config.json len=15
22:52:52-491444 DEBUG    Themes: builtin=8 default=5 external=45
22:52:52-918719 DEBUG    Script: 0.36s ui_tabs
                         C:\StableDiffusion\SDNext\extensions-builtin\stable-diffusion-webui-images-browser\scripts\image_browser.py
22:52:52-928665 DEBUG    Reading: C:\StableDiffusion\SDNext\html\extensions.json len=238
22:52:54-705849 DEBUG    Extension list: processed=228 installed=14 enabled=6 disabled=8 visible=228 hidden=0
22:52:55-105116 INFO     Local URL: http://127.0.0.1:7860/
22:52:55-109074 DEBUG    Gradio registered functions: 1469
22:52:55-112065 INFO     Initializing middleware
22:52:55-117976 DEBUG    Creating API
22:52:55-299913 DEBUG    Scripts setup: ['X/Y/Z Grid:0.008s']
22:52:55-304427 DEBUG    Model metadata: metadata.json no changes
22:52:55-308405 INFO     Select: model="C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix [0aba67f5f6]"
22:52:55-313392 DEBUG    Load model weights: existing=False target=C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors
                         info=None
Loading weights: C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors ---------------------------------- 0.0/2.1 GB -:--:--
22:52:55-370575 DEBUG    Load model: name=C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors dict=True
22:52:55-422437 DEBUG    Desired Torch parameters: dtype=FP16 no-half=False no-half-vae=False upscast=False
22:52:55-427425 INFO     Setting Torch parameters: device=cuda dtype=torch.float16 vae=torch.float16 unet=torch.float16 context=no_grad
                         fp16=True bf16=False
22:52:55-432412 DEBUG    Model dict loaded: {'ram': {'used': 1.0, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
22:52:55-451421 DEBUG    Model config loaded: {'ram': {'used': 1.0, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
22:52:56-048587 INFO     LDM: LatentDiffusion: Running in eps-prediction mode
22:52:56-052538 INFO     LDM: DiffusionWrapper has 859.52 M params.
22:52:56-054531 DEBUG    Model created from config: C:\StableDiffusion\SDNext\configs\v1-inference.yaml
22:52:56-057524 INFO     Autodetect: model="Stable Diffusion" class=StableDiffusionPipeline
                         file="C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors" size=2034MB
22:52:56-062247 DEBUG    Model weights loading: {'ram': {'used': 1.97, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
22:52:57-419618 DEBUG    Model weights loaded: {'ram': {'used': 8.98, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
22:52:58-156856 DEBUG    Model weights moved: {'ram': {'used': 8.84, 'total': 47.96}, 'gpu': {'used': 3.08, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
22:52:58-169984 INFO     Cross-attention: optimization=Sub-quadratic options=[]
22:52:58-581274 INFO     Loaded embeddings: loaded=49 skipped=0 time=0.40s
22:52:58-590291 INFO     Model loaded in 3.27s { load=0.06s config=0.08s create=0.61s apply=0.87s vae=0.48s move=0.73s embeddings=0.41s }
22:52:58-829618 DEBUG    gc: collected=5434 device=cuda {'ram': {'used': 8.05, 'total': 47.96}, 'gpu': {'used': 3.08, 'total': 8.0},
                         'retries': 0, 'oom': 0}
22:52:58-836570 INFO     Model load finished: {'ram': {'used': 8.05, 'total': 47.96}, 'gpu': {'used': 3.08, 'total': 8.0}, 'retries': 0,
                         'oom': 0} cached=1
22:52:58-940320 DEBUG    Saving: config.json len=3036
22:52:58-945280 DEBUG    Unused settings: ['realesrgan_enabled_models', 'state_txt2img', 'control_net_allow_script_control',
                         'upscaling_max_images_in_cache']
22:52:58-950266 INFO     Startup time: 21.62s { torch=8.54s gradio=1.01s diffusers=0.05s libraries=1.80s extensions=1.81s models=0.22s
                         face-restore=0.33s upscalers=0.17s ui-extra-networks=0.66s ui-txt2img=0.11s ui-img2img=0.06s ui-train=0.20s
                         ui-settings=0.20s ui-extensions=2.17s ui-defaults=0.08s launch=0.30s api=0.12s app-started=0.07s checkpoint=3.64s
                         }
22:53:59-600627 DEBUG    Server alive=True jobs=1 requests=2 uptime=72s memory used=5.83 total=47.96 idle
srcrist commented 11 months ago

I am also experiencing this issue. It will start correctly with a completely fresh install if I delete the configs. As soon as I set my model and output directories and save the settings, it will fail to load. If this helps. This is true even if a single checkpoint is the only thing in the model folder. I've tested it a few ways now.

EKKIV0K commented 11 months ago

by the way i hate the new WebUI ^^, is there a way to back to the older disposition :/

srcrist commented 11 months ago

You should take random questions to the discord or something and out of the github issues, please.

EKKIV0K commented 11 months ago

@srcrist im sorry ? but i have the same problem too, the webui doesn't start cause the same problem mentioned on this thread. and .... i don't want to be rude but for the new webui it was not a question it's a fact i don't like it, there is too many changes on one go lol. No need discord to say that ^^

vladmandic commented 11 months ago

@EKKIV0K you can stay on the old version indefinitely, but don't expect and future work of fixes for any issues. methods to pick exact version are well known.

@srcrist can you check what's in your extensions-builtin?

btw, if my hunch is right, staying on old version of sdnext will not solve the hang and it has nothing to do with latest commit.

srcrist commented 11 months ago

@vladmandic Absolutely. I had to run for awhile, but I'll check as soon as i get home.

nCoderGit commented 11 months ago

I think I found the issue: When I add "sd-webui-agent-scheduler" to "disabled_extensions" (config.json), the server starts as it should!

vladmandic commented 11 months ago

really? i have no issues with that extension at all... but i did have quite a few issues with multidiffusion since it was updated 3 days ago.

nCoderGit commented 11 months ago

Yup.

Running webui.bat --debug with

"disabled_extensions": [
    "sd-webui-agent-scheduler"
  ],
00:56:06-346162 DEBUG    Extensions all: ['sd-canvas-editor', 'sd-webui-fabric', 'sd-webui-supermerger', 'sdweb-merge-board',
                         'stable-diffusion-NPW', 'stable-diffusion-webui-embedding-merge', 'stable-diffusion-webui-state']
00:56:06-772301 DEBUG    Running extension installer: C:\StableDiffusion\SDNext\extensions\sd-webui-supermerger\install.py
00:56:20-618153 INFO     Extension installed packages: sd-webui-supermerger ['threadpoolctl==3.2.0', 'joblib==1.3.2',
                         'scikit-learn==1.3.1']
00:56:21-307034 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding', 'sd-extension-chainner',
                         'sd-extension-system-info', 'sd-webui-controlnet', 'stable-diffusion-webui-images-browser',
                         'stable-diffusion-webui-rembg', 'SwinIR', 'sd-canvas-editor', 'sd-webui-fabric', 'sd-webui-supermerger',
                         'sdweb-merge-board', 'stable-diffusion-NPW', 'stable-diffusion-webui-embedding-merge',
                         'stable-diffusion-webui-state']
[...]
00:56:39-772673 DEBUG    Loaded styles: folder=models\styles items=295
00:56:39-780648 DEBUG    Creating UI
00:56:39-907414 INFO     Loading UI theme: name=black-teal style=Auto base=style.css
00:56:40-407059 DEBUG    Extra networks: page='model' items=105 subdirs=1 tab=txt2img dirs=['..\\_SD_Models\\Checkpoints',
                         '..\\SD_Models\\Diffusers', 'C:\\StableDiffusion\\SDNext\\models\\Stable-diffusion'] time=0.17s
00:56:40-435981 DEBUG    Extra networks: page='style' items=295 subdirs=2 tab=txt2img dirs=['models\\styles', 'html'] time=0.02s
00:56:40-446944 DEBUG    Extra networks: page='embedding' items=33 subdirs=2 tab=txt2img dirs=['..\\_SD_Models\\TextualEmbeddings']
                         time=0.09s
00:56:40-455821 DEBUG    Extra networks: page='hypernetwork' items=0 subdirs=0 tab=txt2img dirs=['models\\hypernetworks'] time=0.0s
00:56:40-465133 DEBUG    Extra networks: page='vae' items=3 subdirs=1 tab=txt2img dirs=['..\\_SD_Models\\VAE'] time=0.02s
00:56:40-489862 DEBUG    Extra networks: page='lora' items=270 subdirs=5 tab=txt2img dirs=['C:\\StableDiffusion\\_SD_Models\\Lora',
                         'C:\\StableDiffusion\\_SD_Models\\LyCORIS'] time=0.44s
00:56:40-499524 DEBUG    Extra networks: page='lycoris' items=10 subdirs=3 tab=txt2img dirs=['C:\\StableDiffusion\\_SD_Models\\LyCORIS']
                         time=0.05s
00:56:40-886993 DEBUG    Reading: ui-config.json len=15
00:56:41-005654 DEBUG    Themes: builtin=8 default=5 external=45
00:56:41-586736 DEBUG    Script: 0.48s ui_tabs
                         C:\StableDiffusion\SDNext\extensions-builtin\stable-diffusion-webui-images-browser\scripts\image_browser.py
00:56:41-903499 DEBUG    Script: 0.29s ui_tabs C:\StableDiffusion\SDNext\extensions\sd-webui-supermerger\scripts\supermerger.py
00:56:41-980997 DEBUG    Reading: C:\StableDiffusion\SDNext\html\extensions.json len=238
00:56:44-844079 DEBUG    Extension list: processed=228 installed=21 enabled=20 disabled=1 visible=228 hidden=0
00:56:45-410744 INFO     Local URL: http://127.0.0.1:7860/
00:56:45-414506 DEBUG    Gradio registered functions: 2299
00:56:45-415912 INFO     Initializing middleware
00:56:45-422987 DEBUG    Creating API
00:56:45-779828 DEBUG    Script: 0.12s app_started C:\StableDiffusion\SDNext\extensions\sd-webui-supermerger\scripts\GenParamGetter.py
00:56:45-783028 DEBUG    Scripts setup: ['X/Y/Z Grid:0.01s', 'Tiled Diffusion:0.046s', 'ControlNet:0.091s', 'FABRIC:0.015s']
00:56:45-786021 DEBUG    Model metadata: metadata.json no changes
00:56:45-790011 INFO     Select: model="C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix [0aba67f5f6]"
00:56:45-794018 DEBUG    Load model weights: existing=False target=C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors
                         info=None
Loading weights: C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors ---------------------------------- 0.0/2.1 GB -:--:--
00:56:45-852843 DEBUG    Load model: name=C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors dict=True
00:56:45-904775 DEBUG    Desired Torch parameters: dtype=FP16 no-half=False no-half-vae=False upscast=False
00:56:45-910646 INFO     Setting Torch parameters: device=cuda dtype=torch.float16 vae=torch.float16 unet=torch.float16 context=no_grad
                         fp16=True bf16=False
00:56:45-915645 DEBUG    Model dict loaded: {'ram': {'used': 1.12, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
00:56:45-936607 DEBUG    Model config loaded: {'ram': {'used': 1.12, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
00:56:46-561517 INFO     LDM: LatentDiffusion: Running in eps-prediction mode
00:56:46-563491 INFO     LDM: DiffusionWrapper has 859.52 M params.
00:56:46-566284 DEBUG    Model created from config: C:\StableDiffusion\SDNext\configs\v1-inference.yaml
00:56:46-569610 INFO     Autodetect: model="Stable Diffusion" class=StableDiffusionPipeline
                         file="C:\StableDiffusion\_SD_Models\Checkpoints\NEW_RaeToonMix.safetensors" size=2034MB
00:56:46-575263 DEBUG    Model weights loading: {'ram': {'used': 2.08, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
00:56:48-039829 DEBUG    Model weights loaded: {'ram': {'used': 9.09, 'total': 47.96}, 'gpu': {'used': 1.05, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
00:56:48-785033 DEBUG    Model weights moved: {'ram': {'used': 8.95, 'total': 47.96}, 'gpu': {'used': 3.08, 'total': 8.0}, 'retries': 0,
                         'oom': 0}
00:56:48-798996 INFO     Cross-attention: optimization=Sub-quadratic options=[]
00:56:49-218965 INFO     Loaded embeddings: loaded=49 skipped=0 time=0.41s
00:56:49-230307 INFO     Model loaded in 3.43s { load=0.06s config=0.08s create=0.63s apply=0.97s vae=0.48s move=0.74s embeddings=0.42s }
00:56:49-565338 DEBUG    gc: collected=7996 device=cuda {'ram': {'used': 8.16, 'total': 47.96}, 'gpu': {'used': 3.08, 'total': 8.0},
                         'retries': 0, 'oom': 0}
00:56:49-574845 INFO     Model load finished: {'ram': {'used': 8.16, 'total': 47.96}, 'gpu': {'used': 3.08, 'total': 8.0}, 'retries': 0,
                         'oom': 0} cached=1
00:56:49-687598 DEBUG    Saving: config.json len=2937
00:56:49-692585 DEBUG    Unused settings: ['realesrgan_enabled_models', 'upscaling_max_images_in_cache']
00:56:49-696574 INFO     Startup time: 28.34s { torch=8.72s gradio=0.88s diffusers=0.05s libraries=1.98s extensions=6.03s models=0.22s
                         face-restore=0.32s upscalers=0.19s ui-extra-networks=0.72s ui-txt2img=0.13s ui-img2img=0.16s ui-settings=0.21s
                         ui-extensions=3.79s ui-defaults=0.12s launch=0.43s api=0.14s app-started=0.23s checkpoint=3.90s }
vladmandic commented 11 months ago

hmmm, that's really strange and i cannot reproduce nor i have much to analyze. i'll dig around see if i can find something.

EKKIV0K commented 11 months ago

@vladmandic @nCoderGit hum strange cause on my side i deleted agent scheduler from ext bulltin and the problem still... i don't think it's related to that ...

EKKIV0K commented 11 months ago

Update @vladmandic in fact he is right it worked ! but the message still the same it says ->

WARNING Using Gradio default theme which is not optimized for SD.Next

really strange :o

mygithubaccount222 commented 11 months ago

I think I found the issue: When I add "sd-webui-agent-scheduler" to "disabled_extensions" (config.json), the server starts as it should!

i can confirm this also worked for me, still getting the Gradio error but it works now

nCoderGit commented 11 months ago

What's strange is that python.exe sits at 25% CPU usage.. Does loading the sd-webui-agent-scheduler extension somehow create an infinite loop because it depends on your old implementation of the gradio_theme variable or something like that?

vladmandic commented 11 months ago

not at all. if anything that obvious were the case, i'd be able to reproduce easily.

StealthC commented 11 months ago

I had this problem in a fresh install using an old config.json, I removed the old config.json (sorry didn't make a backup to see what changed) and it worked after that.

srcrist commented 11 months ago

I had this problem in a fresh install using an old config.json, I removed the old config.json (sorry didn't make a backup to see what changed) and it worked after that.

This was the case for me as well, but you may find that it breaks again once you set your model paths.

StealthC commented 11 months ago

I had this problem in a fresh install using an old config.json, I removed the old config.json (sorry didn't make a backup to see what changed) and it worked after that.

This was the case for me as well, but you may find that it breaks again once you set your model paths.

Very true, I'm using relative paths (it worked before), like "..\sd.models\stable-diffusion" and it's now stuck at same place!

srcrist commented 11 months ago

I am also using relative paths. Maybe it's important. Mine are ../models etc.

StealthC commented 11 months ago

I am also using relative paths. Maybe it's important. Mine are ../models etc.

Thats it! If I put absolute paths (like "F:\IA\sd.models\stable-diffusion" I'm in windows btw) it works. It's somehow about relative paths

srcrist commented 11 months ago

That's great news. At least we have a workaround that doesn't involve disabling extensions while Vlad investigates.

EDIT: I can also confirm that editing the config.json to use absolute paths instead of relative paths allows SD.Next to start without hanging, even with sd-webui-agent-scheduler enabled.

vladmandic commented 11 months ago

now that i know what to look for, fix should probably be tomorrow.

nCoderGit commented 11 months ago

I tried to narrow it down as best as I could, since you couldn't repro it on your end.

Apparently the whole thing gets stuck when the txt2img_generate button is created O.ò

No clue why this only happens with sd-webui-agent-scheduler enabled, though. Or what relative vs absolute paths could have to do with it.


(ui.py)

def create_toprow(is_img2img):
    id_part = "img2img" if is_img2img else "txt2img"
    with gr.Row(elem_id=f"{id_part}_toprow", variant="compact"):
        with gr.Column(elem_id=f"{id_part}_prompt_container", scale=6):
            with gr.Row():
                with gr.Column(scale=80):
                    with gr.Row():
                        prompt = gr.Textbox(elem_id=f"{id_part}_prompt", label="Prompt", show_label=False, lines=3, placeholder="Prompt", elem_classes=["prompt"])
            with gr.Row():
                with gr.Column(scale=80):
                    with gr.Row():
                        negative_prompt = gr.Textbox(elem_id=f"{id_part}_neg_prompt", label="Negative prompt", show_label=False, lines=3, placeholder="Negative prompt", elem_classes=["prompt"])
        button_interrogate = None
        button_deepbooru = None
        if is_img2img:
            with gr.Column(scale=1, elem_classes="interrogate-col"):
                button_interrogate = gr.Button('Interrogate\nCLIP', elem_id="interrogate")
                button_deepbooru = gr.Button('Interrogate\nDeepBooru', elem_id="deepbooru")
        with gr.Column(scale=1, elem_id=f"{id_part}_actions_column"):
            log.debug(f"### N ### create_toprow#244")
            with gr.Row(elem_id=f"{id_part}_generate_box"):
                log.debug(f"### N ### create_toprow#246 ({id_part})")
                submit = gr.Button('Generate', elem_id=f"{id_part}_generate", variant='primary')
                log.debug(f"### N ### create_toprow#248 Button created.")
            with gr.Row(elem_id=f"{id_part}_generate_line2"):
                [...]

Last lines of console output:

[...]
04:58:25-876152 INFO     Extensions time: 5.12s { a1111-sd-webui-lycoris=1.44s clip-interrogator-ext=0.08s Lora=0.32s
                         sd-extension-chainner=0.12s sd-extension-system-info=0.06s sd-webui-agent-scheduler=0.52s
                         sd-webui-controlnet=0.76s stable-diffusion-webui-images-browser=0.15s stable-diffusion-webui-rembg=1.15s
                         sd-canvas-editor=0.12s sdweb-merge-board=0.07s stable-diffusion-webui-embedding-merge=0.18s
                         stable-diffusion-webui-state=0.13s }
04:58:26-037975 DEBUG    Reading: html/upscalers.json len=4
04:58:26-044950 DEBUG    Loaded upscalers: total=45 downloaded=10 user=4 ['None', 'Lanczos', 'Nearest', 'ChaiNNer', 'ESRGAN', 'LDSR',
                         'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
04:58:26-058893 DEBUG    Loaded styles: folder=models\styles items=295
04:58:26-065630 DEBUG    Creating UI
04:58:26-071597 DEBUG    ### N ### start_ui#231, before shared.demo = modules.ui.create_ui(timer.startup)
04:58:26-181973 INFO     Loading UI theme: name=black-teal style=Auto base=style.css
04:58:26-265618 DEBUG    ### N ### create_toprow#244
04:58:26-269487 DEBUG    ### N ### create_toprow#246 (txt2img)

_ ► Note that ### N ### create_toprow#248 Button created. doesn't show up in the log.

vladmandic commented 11 months ago

i've pushed an update, can you try? if issue persists, please set env variable SD_EN_DEBUG=true and re-run and post log (it will add more verbose output in the area which should be executing immediately after loading ui theme)

@nCoderGit good analysis, but i'm affraid its going in a wrong direction - this looks like a thread lock, so what you found is that it simply hangs on what happens to be a creation of a very first gradio control while its waiting for thread lock to be released.

StealthC commented 11 months ago

At least at my end, it still hangs up after update (if I'm using relative paths) I've noticed that the problem is only with "ckpt_dir" config, the other paths can be relative and it works.

My log (with relative path for ckpt_dir):

Using VENV: F:\IA\sd.next\venv
11:41:42-072800 DEBUG    Logger: file=F:\IA\sd.next\sdnext.log level=10 size=0 mode=create
11:41:42-075815 INFO     Starting SD.Next
11:41:42-076876 INFO     Python 3.10.11 on Windows
11:41:42-282692 INFO     Version: app=sd.next updated=2023-10-19 hash=e0c92b10
                         url=https://github.com/vladmandic/automatic/tree/master
11:41:43-048043 INFO     Latest published version: 11c073a3caf3933d7a506ee23252f90610379810 2023-10-19T14:37:03Z
11:41:43-067850 INFO     Platform: arch=AMD64 cpu=AMD64 Family 25 Model 97 Stepping 2, AuthenticAMD system=Windows
                         release=Windows-10-10.0.22621-SP0 python=3.10.11
11:41:43-069692 DEBUG    Setting environment tuning
11:41:43-069692 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False openvino=False
11:41:43-071032 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True openvino=True
11:41:43-073627 INFO     nVidia CUDA toolkit detected: nvidia-smi present
11:41:43-209704 DEBUG    Repository update time: Thu Oct 19 11:19:46 2023
11:41:43-210752 INFO     Verifying requirements
11:41:43-218631 INFO     Verifying packages
11:41:43-219670 INFO     Verifying repositories
11:41:43-314630 DEBUG    Submodule: F:\IA\sd.next\repositories\stable-diffusion-stability-ai / main
11:41:44-652737 DEBUG    Submodule: F:\IA\sd.next\repositories\taming-transformers / master
11:41:47-245614 DEBUG    Submodule: F:\IA\sd.next\repositories\BLIP / main
11:41:48-118845 INFO     Verifying submodules
11:41:50-602730 DEBUG    Submodule: extensions-builtin/clip-interrogator-ext / main
11:41:50-701717 DEBUG    Submodule: extensions-builtin/sd-extension-chainner / main
11:41:50-799078 DEBUG    Submodule: extensions-builtin/sd-extension-system-info / main
11:41:50-895815 DEBUG    Submodule: extensions-builtin/sd-webui-agent-scheduler / main
11:41:50-993829 DEBUG    Submodule: extensions-builtin/sd-webui-controlnet / main
11:41:51-092962 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-images-browser / main
11:41:51-191753 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-rembg / master
11:41:51-297957 DEBUG    Submodule: modules/lora / main
11:41:51-398118 DEBUG    Submodule: wiki / master
11:41:51-557857 DEBUG    Installed packages: 241
11:41:51-558857 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-extension-chainner',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
11:41:51-660797 DEBUG    Running extension installer: F:\IA\sd.next\extensions-builtin\clip-interrogator-ext\install.py
11:41:57-377738 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\sd-extension-system-info\install.py
11:41:57-769138 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\sd-webui-agent-scheduler\install.py
11:41:58-183713 DEBUG    Running extension installer: F:\IA\sd.next\extensions-builtin\sd-webui-controlnet\install.py
11:41:58-537126 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\stable-diffusion-webui-images-browser\install.py
11:41:58-904034 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\stable-diffusion-webui-rembg\install.py
11:41:59-399924 DEBUG    Extensions all: ['sd-civitai-browser-plus', 'sd-webui-inpaint-anything',
                         'sd-webui-segment-anything']
11:41:59-400973 DEBUG    Running extension installer: F:\IA\sd.next\extensions\sd-civitai-browser-plus\install.py
11:41:59-763082 DEBUG    Running extension installer: F:\IA\sd.next\extensions\sd-webui-inpaint-anything\install.py
11:42:03-226196 DEBUG    Running extension installer: F:\IA\sd.next\extensions\sd-webui-segment-anything\install.py
11:42:03-567982 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-extension-chainner',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR',
                         'sd-civitai-browser-plus', 'sd-webui-inpaint-anything', 'sd-webui-segment-anything']
11:42:03-570062 INFO     Verifying requirements
11:42:03-577998 DEBUG    Setup complete without errors: 1697726524
11:42:03-582020 INFO     Extension preload: {'extensions-builtin': 0.0, 'extensions': 0.0}
11:42:03-583024 DEBUG    Starting module: <module 'webui' from 'F:\\IA\\sd.next\\webui.py'>
11:42:03-584591 INFO     Command line args: ['--debug'] debug=True
11:42:08-681111 DEBUG    Loaded packages: torch=2.2.0.dev20230922+cu121 diffusers=0.21.4 gradio=3.43.2
11:42:09-242375 DEBUG    Reading: config.json len=29
11:42:09-243886 INFO     Engine: backend=Backend.DIFFUSERS compute=cuda mode=no_grad device=cuda
                         cross-optimization="Scaled-Dot-Product"
11:42:09-309627 INFO     Device: device=NVIDIA GeForce RTX 4090 n=1 arch=sm_90 cap=(8, 9) cuda=12.1 cudnn=8904
                         driver=545.84
11:42:09-972870 DEBUG    Entering start sequence
11:42:09-975370 DEBUG    Initializing
11:42:09-977370 INFO     Available VAEs: ..\sd.models\vae items=5
11:42:09-978370 INFO     Diffusers disabling uncompatible extensions: ['sd-webui-controlnet',
                         'multidiffusion-upscaler-for-automatic1111', 'a1111-sd-webui-lycoris']
11:42:09-980931 DEBUG    Scanning diffusers cache: models\Diffusers models\Diffusers items=3 time=0.00s
11:42:09-984506 DEBUG    Reading: metadata.json len=132
11:42:09-986013 DEBUG    Reading: cache.json len=2
11:42:09-987068 INFO     Available models: ..\sd.models\stable-diffusion items=18 time=0.01s
11:42:10-027338 DEBUG    Loading extensions
11:42:15-292516 INFO     Extension: script='extensions\sd-civitai-browser-plus\scripts\civitai_api.py' Aria2 RPC started
11:42:17-100929 INFO     Extensions time: 7.07s { clip-interrogator-ext=0.70s Lora=0.18s sd-extension-chainner=0.08s
                         sd-webui-agent-scheduler=0.34s stable-diffusion-webui-images-browser=0.09s
                         stable-diffusion-webui-rembg=0.82s sd-civitai-browser-plus=4.13s
                         sd-webui-inpaint-anything=0.39s sd-webui-segment-anything=0.32s }
11:42:17-245651 DEBUG    Reading: html/upscalers.json len=4
11:42:17-248613 DEBUG    Loaded upscalers: total=45 downloaded=5 user=2 ['None', 'Lanczos', 'Nearest', 'ChaiNNer',
                         'ESRGAN', 'LDSR', 'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
11:42:17-260537 DEBUG    Loaded styles: folder=models\styles items=321
11:42:17-262545 INFO     EN register-page: <modules.ui_extra_networks_checkpoints.ExtraNetworksPageCheckpoints object at
                         0x00000140B0AB1270>
11:42:17-264052 INFO     EN register-page: <modules.ui_extra_networks_styles.ExtraNetworksPageStyles object at
                         0x00000140B0AB0A30>
11:42:17-265055 INFO     EN register-page:
                         <modules.ui_extra_networks_textual_inversion.ExtraNetworksPageTextualInversion object at
                         0x00000140B0AB0B20>
11:42:17-265607 INFO     EN register-page: <modules.ui_extra_networks_hypernets.ExtraNetworksPageHypernetworks object at
                         0x00000140B0AFD7E0>
11:42:17-266714 INFO     EN register-page: <modules.ui_extra_networks_vae.ExtraNetworksPageVAEs object at
                         0x00000140B0AFD930>
11:42:17-267771 DEBUG    Creating UI
11:42:17-268775 INFO     EN register-page: <ui_extra_networks_lora.ExtraNetworksPageLora object at 0x00000140AA7B9AE0>
11:42:17-872384 INFO     Loading UI theme: name=black-teal style=Auto base=style.css

and after switching the relative path to absolute (working):

F:\IA\sd.next>.\webui.bat --debug
Using VENV: F:\IA\sd.next\venv
11:45:37-320728 DEBUG    Logger: file=F:\IA\sd.next\sdnext.log level=10 size=0 mode=create
11:45:37-323880 INFO     Starting SD.Next
11:45:37-324884 INFO     Python 3.10.11 on Windows
11:45:37-528743 INFO     Version: app=sd.next updated=2023-10-19 hash=e0c92b10
                         url=https://github.com/vladmandic/automatic/tree/master
11:45:39-313998 INFO     Latest published version: 11c073a3caf3933d7a506ee23252f90610379810 2023-10-19T14:37:03Z
11:45:39-332905 INFO     Platform: arch=AMD64 cpu=AMD64 Family 25 Model 97 Stepping 2, AuthenticAMD system=Windows
                         release=Windows-10-10.0.22621-SP0 python=3.10.11
11:45:39-335033 DEBUG    Setting environment tuning
11:45:39-336038 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False openvino=False
11:45:39-337557 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True openvino=True
11:45:39-339701 INFO     nVidia CUDA toolkit detected: nvidia-smi present
11:45:39-476124 DEBUG    Repository update time: Thu Oct 19 11:19:46 2023
11:45:39-476624 INFO     Verifying requirements
11:45:39-484581 INFO     Verifying packages
11:45:39-486711 INFO     Verifying repositories
11:45:39-579568 DEBUG    Submodule: F:\IA\sd.next\repositories\stable-diffusion-stability-ai / main
11:45:40-577724 DEBUG    Submodule: F:\IA\sd.next\repositories\taming-transformers / master
11:45:43-015884 DEBUG    Submodule: F:\IA\sd.next\repositories\BLIP / main
11:45:43-821762 INFO     Verifying submodules
11:45:46-292686 DEBUG    Submodule: extensions-builtin/clip-interrogator-ext / main
11:45:46-388629 DEBUG    Submodule: extensions-builtin/sd-extension-chainner / main
11:45:46-483919 DEBUG    Submodule: extensions-builtin/sd-extension-system-info / main
11:45:46-581052 DEBUG    Submodule: extensions-builtin/sd-webui-agent-scheduler / main
11:45:46-684653 DEBUG    Submodule: extensions-builtin/sd-webui-controlnet / main
11:45:46-784763 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-images-browser / main
11:45:46-886877 DEBUG    Submodule: extensions-builtin/stable-diffusion-webui-rembg / master
11:45:46-990845 DEBUG    Submodule: modules/lora / main
11:45:47-088892 DEBUG    Submodule: wiki / master
11:45:47-259979 DEBUG    Installed packages: 241
11:45:47-261563 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-extension-chainner',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
11:45:47-386747 DEBUG    Running extension installer: F:\IA\sd.next\extensions-builtin\clip-interrogator-ext\install.py
11:45:53-003447 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\sd-extension-system-info\install.py
11:45:53-340000 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\sd-webui-agent-scheduler\install.py
11:45:53-683985 DEBUG    Running extension installer: F:\IA\sd.next\extensions-builtin\sd-webui-controlnet\install.py
11:45:54-042903 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\stable-diffusion-webui-images-browser\install.py
11:45:54-392977 DEBUG    Running extension installer:
                         F:\IA\sd.next\extensions-builtin\stable-diffusion-webui-rembg\install.py
11:45:54-847046 DEBUG    Extensions all: ['sd-civitai-browser-plus', 'sd-webui-inpaint-anything',
                         'sd-webui-segment-anything']
11:45:54-847550 DEBUG    Running extension installer: F:\IA\sd.next\extensions\sd-civitai-browser-plus\install.py
11:45:55-180915 DEBUG    Running extension installer: F:\IA\sd.next\extensions\sd-webui-inpaint-anything\install.py
11:45:58-285501 DEBUG    Running extension installer: F:\IA\sd.next\extensions\sd-webui-segment-anything\install.py
11:45:58-622789 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-extension-chainner',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR',
                         'sd-civitai-browser-plus', 'sd-webui-inpaint-anything', 'sd-webui-segment-anything']
11:45:58-624847 INFO     Verifying requirements
11:45:58-632977 DEBUG    Setup complete without errors: 1697726759
11:45:58-638000 INFO     Extension preload: {'extensions-builtin': 0.0, 'extensions': 0.0}
11:45:58-638881 DEBUG    Starting module: <module 'webui' from 'F:\\IA\\sd.next\\webui.py'>
11:45:58-639939 INFO     Command line args: ['--debug'] debug=True
11:46:03-820572 DEBUG    Loaded packages: torch=2.2.0.dev20230922+cu121 diffusers=0.21.4 gradio=3.43.2
11:46:04-437637 DEBUG    Reading: config.json len=29
11:46:04-439739 INFO     Engine: backend=Backend.DIFFUSERS compute=cuda mode=no_grad device=cuda
                         cross-optimization="Scaled-Dot-Product"
11:46:04-515783 INFO     Device: device=NVIDIA GeForce RTX 4090 n=1 arch=sm_90 cap=(8, 9) cuda=12.1 cudnn=8904
                         driver=545.84
11:46:05-279668 DEBUG    Entering start sequence
11:46:05-281771 DEBUG    Initializing
11:46:05-283872 INFO     Available VAEs: ..\sd.models\vae items=5
11:46:05-285391 INFO     Diffusers disabling uncompatible extensions: ['sd-webui-controlnet',
                         'multidiffusion-upscaler-for-automatic1111', 'a1111-sd-webui-lycoris']
11:46:05-286937 DEBUG    Scanning diffusers cache: models\Diffusers models\Diffusers items=3 time=0.00s
11:46:05-291307 DEBUG    Reading: metadata.json len=132
11:46:05-292823 DEBUG    Reading: cache.json len=2
11:46:05-294862 INFO     Available models: F:\IA\sd.models\stable-diffusion items=18 time=0.01s
11:46:05-347253 DEBUG    Loading extensions
11:46:10-709778 INFO     Extension: script='extensions\sd-civitai-browser-plus\scripts\civitai_api.py' Aria2 RPC started
11:46:12-544605 INFO     Extensions time: 7.20s { clip-interrogator-ext=0.68s Lora=0.21s sd-extension-chainner=0.08s
                         sd-extension-system-info=0.05s sd-webui-agent-scheduler=0.34s
                         stable-diffusion-webui-images-browser=0.12s stable-diffusion-webui-rembg=0.86s
                         sd-civitai-browser-plus=4.15s sd-webui-inpaint-anything=0.36s sd-webui-segment-anything=0.34s }
11:46:12-655555 DEBUG    Reading: html/upscalers.json len=4
11:46:12-659621 DEBUG    Loaded upscalers: total=45 downloaded=5 user=2 ['None', 'Lanczos', 'Nearest', 'ChaiNNer',
                         'ESRGAN', 'LDSR', 'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
11:46:12-671656 DEBUG    Loaded styles: folder=models\styles items=321
11:46:12-674309 INFO     EN register-page: <modules.ui_extra_networks_checkpoints.ExtraNetworksPageCheckpoints object at
                         0x00000131B03A5330>
11:46:12-675821 INFO     EN register-page: <modules.ui_extra_networks_styles.ExtraNetworksPageStyles object at
                         0x00000131B03A4AF0>
11:46:12-676825 INFO     EN register-page:
                         <modules.ui_extra_networks_textual_inversion.ExtraNetworksPageTextualInversion object at
                         0x00000131B03A4BE0>
11:46:12-677923 INFO     EN register-page: <modules.ui_extra_networks_hypernets.ExtraNetworksPageHypernetworks object at
                         0x00000131B03E98A0>
11:46:12-678998 INFO     EN register-page: <modules.ui_extra_networks_vae.ExtraNetworksPageVAEs object at
                         0x00000131B03E99F0>
11:46:12-680553 DEBUG    Creating UI
11:46:12-680553 INFO     EN register-page: <ui_extra_networks_lora.ExtraNetworksPageLora object at 0x00000131AA01DBA0>
11:46:13-015760 INFO     Loading UI theme: name=black-teal style=Auto base=style.css
11:46:13-048201 INFO     EN create-ui: txt2img
11:46:13-051809 INFO     EN create-items: model
11:46:13-052934 INFO     EN create-items: style
11:46:13-054975 INFO     EN create-items: embedding
11:46:13-055980 INFO     EN create-items: hypernetwork
11:46:13-058115 INFO     EN create-items: vae
11:46:13-059717 INFO     EN create-items: lora
11:46:13-122046 INFO     EN create-page: model
11:46:13-123566 INFO     EN list-dir: F:\IA\sd.models\stable-diffusion\sd15
11:46:13-125061 INFO     EN list-dir: F:\IA\sd.models\stable-diffusion\sdxl
11:46:13-126634 INFO     EN list-dir: F:\IA\sd.models\stable-diffusion
11:46:13-127644 INFO     EN list-dir: F:\IA\sd.next\models\Diffusers
11:46:13-128142 INFO     EN list-dir: F:\IA\sd.next\models\Stable-diffusion
11:46:13-128641 INFO     EN create-items: model
11:46:13-130656 DEBUG    Extra networks: page='model' items=18 subdirs=4 tab=txt2img
                         dirs=['F:\\IA\\sd.models\\stable-diffusion', 'models\\Diffusers',
                         'F:\\IA\\sd.next\\models\\Stable-diffusion'] time=0.04s
11:46:13-132163 INFO     EN create-thumb: model
11:46:13-133161 INFO     EN create-page: style
11:46:13-134992 INFO     EN list-dir: F:\IA\sd.next\models\styles\Add Ons
11:46:13-134992 INFO     EN list-dir: F:\IA\sd.next\models\styles\Classic Negative Prompts
11:46:13-136607 INFO     EN list-dir: F:\IA\sd.next\models\styles\Classic Positive Prompts
11:46:13-137610 INFO     EN list-dir: F:\IA\sd.next\models\styles\SDXL Effects
11:46:13-138699 INFO     EN list-dir: F:\IA\sd.next\models\styles\SDXL Specific
11:46:13-139759 INFO     EN list-dir: F:\IA\sd.next\models\styles
11:46:13-140765 INFO     EN list-dir: F:\IA\sd.next\html
11:46:13-141872 INFO     EN create-items: style
11:46:13-153401 DEBUG    Extra networks: page='style' items=321 subdirs=7 tab=txt2img dirs=['models\\styles', 'html']
                         time=0.03s
11:46:13-155970 INFO     EN create-page: embedding
11:46:13-159653 INFO     EN list-dir: F:\IA\sd.models\textual-inversions\negative
11:46:13-161154 INFO     EN list-dir: F:\IA\sd.models\textual-inversions\positive
11:46:13-162678 INFO     EN list-dir: F:\IA\sd.models\textual-inversions\sdxl_negative
11:46:13-163685 INFO     EN list-dir: F:\IA\sd.models\textual-inversions\wallpapers\ö
11:46:13-164693 INFO     EN list-dir: F:\IA\sd.models\textual-inversions\wallpapers
11:46:13-166313 INFO     EN list-dir: F:\IA\sd.models\textual-inversions
11:46:13-167319 INFO     EN create-items: embedding
11:46:13-167822 DEBUG    Extra networks: page='embedding' items=0 subdirs=6 tab=txt2img
                         dirs=['..\\sd.models\\textual-inversions'] time=0.0s
11:46:13-168758 INFO     EN create-page: hypernetwork
11:46:13-170033 INFO     EN list-dir: F:\IA\sd.models\hypernetworks
11:46:13-171543 INFO     EN create-items: hypernetwork
11:46:13-171543 DEBUG    Extra networks: page='hypernetwork' items=0 subdirs=0 tab=txt2img
                         dirs=['..\\sd.models\\hypernetworks'] time=0.0s
11:46:13-172570 INFO     EN create-page: vae
11:46:13-174659 INFO     EN list-dir: F:\IA\sd.models\vae\sd15
11:46:13-175709 INFO     EN list-dir: F:\IA\sd.models\vae\sdxl-vae-fp16-fix
11:46:13-175709 INFO     EN list-dir: F:\IA\sd.models\vae
11:46:13-176757 INFO     EN create-items: vae
11:46:13-177894 DEBUG    Extra networks: page='vae' items=5 subdirs=3 tab=txt2img dirs=['..\\sd.models\\vae'] time=0.01s
11:46:13-178898 INFO     EN create-page: lora
11:46:13-180909 INFO     EN list-dir: F:\IA\sd.models\lora\sd15\characters
11:46:13-181913 INFO     EN list-dir: F:\IA\sd.models\lora\sd15
11:46:13-182419 INFO     EN list-dir: F:\IA\sd.models\lora\sd15-nsfw
11:46:13-182918 INFO     EN list-dir: F:\IA\sd.models\lora\sdxl
11:46:13-183888 INFO     EN list-dir: F:\IA\sd.models\lora\sdxl-nsfw
11:46:13-185549 INFO     EN list-dir: F:\IA\sd.models\lora
11:46:13-186598 INFO     EN list-dir: F:\IA\sd.next\models\LyCORIS
11:46:13-187105 INFO     EN create-items: lora
11:46:13-189116 DEBUG    Extra networks: page='lora' items=39 subdirs=6 tab=txt2img dirs=['..\\sd.models\\lora',
                         'models\\LyCORIS'] time=0.06s
11:46:13-191155 INFO     EN create-thumb: lora
11:46:13-245759 INFO     EN create-ui: img2img
11:46:13-249989 INFO     EN create-items: model
11:46:13-251038 INFO     EN create-items: style
11:46:13-251038 INFO     EN create-items: embedding
11:46:13-252556 INFO     EN create-items: hypernetwork
11:46:13-253632 INFO     EN create-items: vae
11:46:13-253632 INFO     EN create-items: lora
11:46:13-254720 INFO     EN create-page: model
11:46:13-255783 INFO     EN create-page: style
11:46:13-256841 INFO     EN create-page: embedding
11:46:13-257890 INFO     EN create-page: hypernetwork
11:46:13-257890 INFO     EN create-page: vae
11:46:13-258935 INFO     EN create-page: lora
11:46:13-337972 DEBUG    Reading: ui-config.json len=4
11:46:13-362819 DEBUG    Themes: builtin=8 default=5 external=51
11:46:13-675775 DEBUG    Script: 0.24s ui_tabs
                         F:\IA\sd.next\extensions-builtin\stable-diffusion-webui-images-browser\scripts\image_browser.py
11:46:13-721557 DEBUG    Reading: F:\IA\sd.next\html\extensions.json len=238
11:46:14-985968 DEBUG    Extension list: processed=228 installed=16 enabled=13 disabled=3 visible=228 hidden=0
11:46:15-629705 INFO     Local URL: http://127.0.0.1:7860/
11:46:15-630826 DEBUG    Gradio registered functions: 1670
11:46:15-631777 INFO     Initializing middleware
11:46:15-636872 DEBUG    Creating API
11:46:15-806930 INFO     [AgentScheduler] Task queue is empty
11:46:15-807953 INFO     [AgentScheduler] Registering APIs
11:46:15-907391 DEBUG    Scripts setup: ['Segment Anything:0.017s']
11:46:15-908899 DEBUG    Model metadata: metadata.json no changes
11:46:15-910736 INFO     Select: model="sdxl\dreamshaperXL10_alpha2Xl10 [0f1b80cfe8]"
11:46:15-912255 DEBUG    Load model weights: existing=False
                         target=F:\IA\sd.models\stable-diffusion\sdxl\dreamshaperXL10_alpha2Xl10.safetensors info=None
Loading weights: F:\IA\sd.models\stable-diffusion\sdxl\dreamshaperXL10_alpha2Xl10.safetensors ━━━━━━━━ 0.0/6.9   -:--:--
                                                                                                       GB
11:46:16-549026 DEBUG    Desired Torch parameters: dtype=FP16 no-half=False no-half-vae=False upscast=False
11:46:16-550612 INFO     Setting Torch parameters: device=cuda dtype=torch.float16 vae=torch.float16 unet=torch.float16
                         context=no_grad fp16=True bf16=False
11:46:16-552638 INFO     Autodetect: model="Stable Diffusion XL" class=StableDiffusionXLPipeline
                         file="F:\IA\sd.models\stable-diffusion\sdxl\dreamshaperXL10_alpha2Xl10.safetensors" size=6617MB
11:46:22-728646 DEBUG    Setting model: pipeline=StableDiffusionXLPipeline config={'low_cpu_mem_usage': True,
                         'torch_dtype': torch.float16, 'load_connected_pipeline': True, 'variant': 'fp16',
                         'extract_ema': True, 'force_zeros_for_empty_prompt ': True, 'requires_aesthetics_score': False,
                         'use_safetensors': True}
11:46:22-730789 DEBUG    Setting model: enable VAE slicing
11:46:22-731793 DEBUG    Setting model: enable VAE tiling
11:46:22-742195 DEBUG    Setting model VAE: name=None upcast=True
11:46:26-655050 INFO     Loaded embeddings: loaded=4 skipped=78 time=2.17s
11:46:27-007703 DEBUG    gc: collected=11039 device=cuda {'ram': {'used': 2.18, 'total': 63.14}, 'gpu': {'used': 8.56,
                         'total': 23.99}, 'retries': 0, 'oom': 0}
11:46:27-015981 INFO     Loaded model: time=10.74s { load=10.74s } native=1024 {'ram': {'used': 2.18, 'total': 63.14},
                         'gpu': {'used': 8.56, 'total': 23.99}, 'retries': 0, 'oom': 0}
11:46:27-022753 DEBUG    Saving: config.json len=1311
11:46:27-024850 INFO     Startup time: 28.38s { torch=4.73s gradio=0.42s libraries=1.46s extensions=7.20s
                         face-restore=0.05s upscalers=0.12s ui-extra-networks=0.51s ui-img2img=0.07s ui-settings=0.08s
                         ui-extensions=1.60s ui-defaults=0.41s launch=0.22s api=0.06s app-started=0.21s
                         checkpoint=11.12s }
StealthC commented 11 months ago

Trying to help here, I'm debugging the scripts and the hang in my side is on this loop on that function on extensions-builtin\sd-webui-agent-scheduler\scripts\task_scheduler.py

def get_checkpoint_choices():
    checkpoints = list_checkpoint_tiles()

    checkpoint_dirs = defaultdict(lambda: 0)
    for checkpoint in checkpoints:
        checkpoint_dir = os.path.dirname(checkpoint)
        while checkpoint_dir != "":
            checkpoint_dirs[checkpoint_dir] += 1
            checkpoint_dir = os.path.dirname(checkpoint_dir) <-- This line

    choices = checkpoints
    choices.extend([f"{d} ({checkpoint_dirs[d]} checkpoints)" for d in checkpoint_dirs.keys()])
    choices = sorted(choices)

    choices.insert(0, checkpoint_runtime)
    choices.insert(0, checkpoint_current)

    return choices

Edit: If I've understood it, checkpoint_dir starts with the absolute path to checkpoint (even when it passed as relative path in config), and it is going up a folder each loop until it gets to root drive (I'm in windows, so it's like F:\\) and it will repeat forever

Edit 2: when starting with relative paths, the checkpoint value is like 'F:\\IA\\sd.models\\stable-diffusion\\sd15\\cuteyukimixAdorable_midchapter2'

but when correcting to absolute paths, it will be 'sd15\\cuteyukimixAdorable_midchapter2 [a0f5a29a85]'

and works. So I think the cause is when that checkpoint name is created. Also is possible that this problem only occur on Windows because of the way the path is parsed.

srcrist commented 11 months ago

I don't actually have new information to add, but my result was the same as @StealthC when I tested.

vladmandic commented 11 months ago

just to make sure, when you say "relative path", you're refering to relative paths starting with ".."? any other relative path should work just fine (and is in fact default behavior).

StealthC commented 11 months ago

just to make sure, when you say "relative path", you're refering to relative paths starting with ".."? any other relative path should work just fine (and is in fact default behavior).

yes, the problem occurs with relative path to a sibling or parent folder, like "..\sd.models\" relative folders to subfolders apparently still works.

vladmandic commented 11 months ago

just pushed, can you try?

nCoderGit commented 11 months ago

Can confirm.

Test code:

    def after_component(self, component, **_kwargs):
        log.debug(f"### N ### after_component({self}, {component}, ...) called.")
        generate_id = "txt2img_generate" if self.is_txt2img else "img2img_generate"
        neg_id = "txt2img_neg_prompt" if self.is_txt2img else "img2img_neg_prompt"

        if component.elem_id == generate_id:
            self.generate_button = component
            log.debug(f"### N ### (inside after component) after self.generate_button = component")
            if getattr(shared.opts, "queue_button_placement", placement_under_generate) == placement_under_generate:
                log.debug(f"### N ### (inside after component) before self.addenqueue_button(), generate_id={generate_id} .")
                self.add_enqueue_button()
                component.parent.children.pop()
                component.parent.parent.add(self.enqueue_row)
            return

"ckpt_dir": "..\\_SD_Models\\Checkpoints", doesn't work and stops before creating a dropdown.

17:57:33-375180 DEBUG    Loaded styles: folder=models\styles items=295
17:57:33-381152 INFO     EN register-page: <modules.ui_extra_networks_checkpoints.ExtraNetworksPageCheckpoints object at
                         0x000001C5D167C2E0>
17:57:33-386078 INFO     EN register-page: <modules.ui_extra_networks_styles.ExtraNetworksPageStyles object at 0x000001C5D167C0A0>
17:57:33-388072 INFO     EN register-page: <modules.ui_extra_networks_textual_inversion.ExtraNetworksPageTextualInversion object at
                         0x000001C5D167C2B0>
17:57:33-392081 INFO     EN register-page: <modules.ui_extra_networks_hypernets.ExtraNetworksPageHypernetworks object at
                         0x000001C5D16B4A30>
17:57:33-396049 INFO     EN register-page: <modules.ui_extra_networks_vae.ExtraNetworksPageVAEs object at 0x000001C5D16B4B80>
17:57:33-400040 DEBUG    Creating UI
17:57:33-402061 INFO     EN register-page: <ui_extra_networks_lora.ExtraNetworksPageLora object at 0x000001C5CAA80250>
17:57:33-405025 INFO     EN register-page: <ui_extra_networks_lyco.ExtraNetworksPageLyCORIS object at 0x000001C5D16B55D0>
17:57:33-554476 INFO     Loading UI theme: name=black-orange style=Auto base=style.css
17:57:33-631037 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x000001C5D16B5AB0>, textbox, ...) called.
17:57:33-636276 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x000001C5D16B5AB0>, textbox, ...) called.

[end of log]


"ckpt_dir": "C:\\StableDiffusion\\_SD_Models\\Checkpoints", works

17:47:29-620804 DEBUG    Loaded styles: folder=models\styles items=295
17:47:29-627619 INFO     EN register-page: <modules.ui_extra_networks_checkpoints.ExtraNetworksPageCheckpoints object at
                         0x00000172BECE8340>
17:47:29-632832 INFO     EN register-page: <modules.ui_extra_networks_styles.ExtraNetworksPageStyles object at 0x00000172BECE80A0>
17:47:29-636992 INFO     EN register-page: <modules.ui_extra_networks_textual_inversion.ExtraNetworksPageTextualInversion object at
                         0x00000172BECE8310>
17:47:29-640014 INFO     EN register-page: <modules.ui_extra_networks_hypernets.ExtraNetworksPageHypernetworks object at
                         0x00000172BED24A90>
17:47:29-642976 INFO     EN register-page: <modules.ui_extra_networks_vae.ExtraNetworksPageVAEs object at 0x00000172BED24BE0>
17:47:29-647006 DEBUG    Creating UI
17:47:29-650367 INFO     EN register-page: <ui_extra_networks_lora.ExtraNetworksPageLora object at 0x00000172B80D82B0>
17:47:29-653361 INFO     EN register-page: <ui_extra_networks_lyco.ExtraNetworksPageLyCORIS object at 0x00000172BED25630>
17:47:29-795793 INFO     Loading UI theme: name=black-orange style=Auto base=style.css
17:47:29-923932 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x00000172BED25B10>, textbox, ...) called.
17:47:29-930062 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x00000172BED25B10>, textbox, ...) called.
17:47:29-934052 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x00000172BED25B10>, dropdown, ...) called.
17:47:29-936046 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x00000172BED25B10>, button, ...) called.
17:47:29-939037 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x00000172BED25B10>, button, ...) called.
17:47:29-942030 DEBUG    ### N ### (inside after component) after self.generate_button = component
17:47:29-944025 DEBUG    ### N ### (inside after component) before self.addenqueue_button(), generate_id=txt2img_generate .
17:47:29-949015 DEBUG    ### N ### add_enqueuebutton() called.
17:47:29-951006 DEBUG    ### N ### get_checkpoint_choices() called.
17:47:29-953998 DEBUG    ### N ### after_component(<task_scheduler.py.Script object at 0x00000172BED25B10>, dropdown, ...) called.
[...]

[loads successfully]

vladmandic commented 11 months ago

with latest update? its supposed to translate any path starting with .. into absolute path before using it further.

StealthC commented 11 months ago

The problem still occurs, I have seen your fix, I then tried to narrow down more the issue: modules\sd_models.py when parsing the filename and relname, it still uses shared.opts.ckpt_dir directly and not your fixed absolute path, so it never executes the proper logic:

if relname.startswith('..'):
            relname = os.path.abspath(relname)
        if relname.startswith(shared.opts.ckpt_dir):
            relname = rel(filename, shared.opts.ckpt_dir)
        elif relname.startswith(shared.opts.diffusers_dir):
            relname = rel(filename, shared.opts.diffusers_dir)
        elif relname.startswith(model_path):
            relname = rel(filename, model_path)
        elif relname.startswith(script_path):
            relname = rel(filename, script_path)
        elif relname.startswith(app_path):
            relname = rel(filename, app_path)
        else:
            relname = os.path.abspath(relname)
        relname, ext = os.path.splitext(relname)
        ext = ext.lower()[1:]

Edit: Just to be clear, the filename in my example is: F:\\IA\\sd.models\\stable-diffusion\\sd15\\cuteyukimixAdorable_midchapter2.safetensors and the ckpt_dir is: '..\\sd.models\\stable-diffusion' so the logic in if relname.startswith(shared.opts.ckpt_dir): is not executed

if it used the absolute path in fix_path function it will work (just tested it)

by example, this code works:

if relname.startswith(os.path.abspath(shared.opts.ckpt_dir)):
            relname = rel(filename, os.path.abspath(shared.opts.ckpt_dir))
nCoderGit commented 11 months ago

Just did a checkout and pulled the latest version. Still stuck at Loading UI theme [...] when using "ckpt_dir": "..\\_SD_Models\\Checkpoints" and working with absolute ckpt_dir path

srcrist commented 11 months ago

I doubt an additional log would be useful, but I am also still experiencing the same behavior on the latest update as of this comment.

vladmandic commented 11 months ago

ok, i gave up on stumbling and since this is a window-specific issue, i've dusted off my windows installation and reproduced. not it should be actually fixed:

if you want extra debug on path resolving, you can set env variable SD_PATH_DEBUG. (and you can see how it finally correctly resolved relative paths)

13:21:50-586472 INFO     Paths: folder=ckpt_dir original="..\Models\Stable-diffusion" target="F:\Models\Stable-diffusion"
13:21:50-587471 INFO     Paths: folder=diffusers_dir original="models\Diffusers" target="models\Diffusers"
13:21:50-587471 INFO     Paths: folder=vae_dir original="models\VAE" target="models\VAE"
13:21:50-588470 INFO     Paths: folder=lora_dir original="models\Lora" target="models\Lora"
13:21:50-589470 INFO     Paths: folder=embeddings_dir original="models\embeddings" target="models\embeddings"
13:21:50-589470 INFO     Paths: folder=hypernetwork_dir original="models\hypernetworks" target="models\hypernetworks"
13:21:50-590470 INFO     Paths: folder=outdir_txt2img_samples original="outputs\text" target="outputs\text"
13:21:50-591470 INFO     Paths: folder=outdir_img2img_samples original="outputs\image" target="outputs\image"
13:21:50-592470 INFO     Paths: folder=outdir_extras_samples original="outputs\extras" target="outputs\extras"
13:21:50-593471 INFO     Paths: folder=outdir_txt2img_grids original="outputs\grids" target="outputs\grids"
13:21:50-594511 INFO     Paths: folder=outdir_img2img_grids original="outputs\grids" target="outputs\grids"
13:21:50-594511 INFO     Paths: folder=outdir_save original="outputs\save" target="outputs\save"
13:21:50-595511 INFO     Paths: folder=styles_dir original="models\styles" target="models\styles"

fix is in commit 441ca2f6

srcrist commented 11 months ago

Just commenting to confirm that this is now fixed with an update for me. Thanks, Vlad.

nCoderGit commented 11 months ago

Awesome, works like a charm. Thank u :)