vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.36k stars 382 forks source link

[Issue]: Startup failure during torch import #1367

Closed inktomi closed 1 year ago

inktomi commented 1 year ago

Issue Description

Using the code from today, webui.bat will not start up.

2023-06-08 16:31:48,046 | sd | DEBUG | installer | Submodules list: 123d1da15d802823480f8020312ce449523f10e2 extensions-builtin/a1111-sd-webui-lycoris (heads/main)
 9e6bbd9b8931bbe869a8e28e7005b0e13c2efff0 extensions-builtin/clip-interrogator-ext (heads/main)
 70b3c5ea3c9f684d04e7ff59167565974415735c extensions-builtin/multidiffusion-upscaler-for-automatic1111 (sd-webui-v1.1.0-25-g70b3c5e)
 f02cacfc923e8bbf73f25327d722d50c458d66bb extensions-builtin/sd-dynamic-thresholding (heads/master)
 064c856acaed8c73c3a232e39df8c37d0bb9a15c extensions-builtin/sd-extension-system-info (heads/main)
 45dfe5977ea43a14f61f46d572615efe169ca2be extensions-builtin/sd-webui-agent-scheduler (heads/main)
 2598ca9e78f167014c6556cb89dab792a745749f extensions-builtin/sd-webui-controlnet (heads/main)
 b2f6e4cbe507078e3bf2b9d5fce6f83e2e233aa8 extensions-builtin/stable-diffusion-webui-images-browser (heads/main)
 3d9eedbbf0d585207f97d5b21e42f32c0042df70 extensions-builtin/stable-diffusion-webui-rembg (heads/master)
 6417f5d7c183eccd79422f28804f7d7c507e05b9 modules/lora (v0.6.4-104-g6417f5d)
 57d71913bf47a48dc7dd519b89dd30e09c937b83 modules/lycoris (heads/main)
 801130e4c45c815c4b2718a70ae88a2653fd780e wiki (heads/master)
2023-06-08 16:31:48,554 | sd | INFO | installer | Updating submodules
2023-06-08 16:31:55,611 | sd | DEBUG | installer | Installed packages: 241
2023-06-08 16:31:55,611 | sd | DEBUG | installer | Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
2023-06-08 16:31:57,075 | sd | DEBUG | installer | Running extension installer: G:\automatic\extensions-builtin\clip-interrogator-ext\install.py

There is no more relevant logging in any of the log files. I don't see a specific error. I am starting using .\webui.bat --data-dir W:\outputs --models-dir W:\models --insecure --listen --upgrade. I also tried with --reinstall but no change.

Version Platform Description

Windows 11

16:28:53-294336 INFO     Python 3.10.11 on Windows
16:28:53-329870 INFO     Version: 4a3ab4ed Thu Jun 8 16:10:39 2023 -0400
16:28:53-662945 INFO     Setting environment tuning
16:28:53-664947 INFO     nVidia CUDA toolkit detected
16:28:53-665948 INFO     Installing package: torch torchvision --index-url https://download.pytorch.org/whl/cu118
16:29:14-256029 INFO     Torch 2.0.1+cu118
16:29:14-278049 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8700
16:29:14-280051 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 23028 Arch (8, 9) Cores 128
vladmandic commented 1 year ago

Run with --debug to capture more info.

inktomi commented 1 year ago

Here is the full console output with --debug

PS G:\automatic> .\webui.bat --data-dir W:\outputs --models-dir W:\models --insecure --listen --upgrade --debug
Using VENV: G:\automatic\venv
16:47:34-754767 INFO     Starting SD.Next
16:47:34-767281 INFO     Python 3.10.11 on Windows
16:47:34-804315 INFO     Version: 4a3ab4ed Thu Jun 8 16:10:39 2023 -0400
16:47:35-139131 INFO     Setting environment tuning
16:47:35-140132 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False
16:47:35-141133 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True
16:47:35-143134 INFO     nVidia CUDA toolkit detected
16:47:53-855041 INFO     Torch 2.0.1+cu118
16:47:53-880578 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8700
16:47:53-882580 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 23028 Arch (8, 9) Cores 128
16:47:54-033220 DEBUG    Repository update time: Thu Jun  8 13:10:39 2023
16:47:54-034221 INFO     Verifying requirements
16:47:54-046232 INFO     Verifying packages
16:47:54-047233 INFO     Installing repositories
16:47:56-941424 INFO     Installing submodules
16:47:58-025253 DEBUG    Submodules list: 123d1da15d802823480f8020312ce449523f10e2
                         extensions-builtin/a1111-sd-webui-lycoris (heads/main)
                          9e6bbd9b8931bbe869a8e28e7005b0e13c2efff0 extensions-builtin/clip-interrogator-ext (heads/main)                          70b3c5ea3c9f684d04e7ff59167565974415735c
                         extensions-builtin/multidiffusion-upscaler-for-automatic1111 (sd-webui-v1.1.0-25-g70b3c5e)
                          f02cacfc923e8bbf73f25327d722d50c458d66bb extensions-builtin/sd-dynamic-thresholding
                         (heads/master)
                          064c856acaed8c73c3a232e39df8c37d0bb9a15c extensions-builtin/sd-extension-system-info
                         (heads/main)
                          45dfe5977ea43a14f61f46d572615efe169ca2be extensions-builtin/sd-webui-agent-scheduler
                         (heads/main)
                          2598ca9e78f167014c6556cb89dab792a745749f extensions-builtin/sd-webui-controlnet (heads/main)
                          b2f6e4cbe507078e3bf2b9d5fce6f83e2e233aa8
                         extensions-builtin/stable-diffusion-webui-images-browser (heads/main)
                          3d9eedbbf0d585207f97d5b21e42f32c0042df70 extensions-builtin/stable-diffusion-webui-rembg
                         (heads/master)
                          6417f5d7c183eccd79422f28804f7d7c507e05b9 modules/lora (v0.6.4-104-g6417f5d)
                          57d71913bf47a48dc7dd519b89dd30e09c937b83 modules/lycoris (heads/main)
                          801130e4c45c815c4b2718a70ae88a2653fd780e wiki (heads/master)
16:47:58-589788 INFO     Updating submodules
16:48:20-978239 ERROR    Error running git: extensions-builtin/multidiffusion-upscaler-for-automatic1111 / pull origin
                         main --autostash --rebase --force
16:48:20-979240 DEBUG    Git output: fatal: unable to access
                         'https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111/': Failed to connect
                         to github.com port 443 after 21082 ms: Couldn't connect to server
16:48:26-189467 DEBUG    Installed packages: 241
16:48:26-190468 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding',
                         'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
16:48:27-594359 DEBUG    Running extension installer: G:\automatic\extensions-builtin\clip-interrogator-ext\install.py
16:48:49-819832 ERROR    Error running extension installer: G:\automatic\extensions-builtin\clip-interrogator-ext\install.py
16:48:49-820833 DEBUG
16:48:55-057598 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-extension-system-info\install.py
16:48:56-472258 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-webui-agent-scheduler\install.py
16:48:57-825719 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-webui-controlnet\install.py
16:49:00-018369 DEBUG    Running extension installer: G:\automatic\extensions-builtin\seed_travel\install.py
16:49:01-380949 DEBUG    Running extension installer: G:\automatic\extensions-builtin\stable-diffusion-webui-images-browser\install.py
16:49:03-633970 DEBUG    Running extension installer: G:\automatic\extensions-builtin\stable-diffusion-webui-rembg\install.py
16:49:04-736437 DEBUG    Extensions all: ['adetailer', 'sd-webui-deforum']
16:49:05-557785 DEBUG    Running extension installer: W:\outputs\extensions\adetailer\install.py
16:49:07-322834 DEBUG    Running extension installer: W:\outputs\extensions\sd-webui-deforum\install.py
16:49:08-160570 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding',
                         'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'adetailer',
                         'sd-webui-deforum']
16:49:08-162572 INFO     Updating Wiki
16:49:08-724602 WARNING  Setup complete with errors: 2
16:49:08-725603 WARNING  See log file for more details: G:\automatic\sdnext.log
16:49:08-731609 INFO     Extension preload: 0.0s G:\automatic\extensions-builtin
16:49:08-740617 INFO     Extension preload: 0.0s W:\outputs\extensions
16:49:08-742119 DEBUG    Args: {'config': 'W:\\outputs\\config.json', 'ui_config': 'W:\\outputs\\ui-config.json', 'medvram': False,
                         'lowvram': False, 'ckpt': None, 'vae': None, 'data_dir': 'W:\\outputs', 'models_dir': 'W:\\models', 'allow_code':
                         False, 'share': False, 'insecure': True, 'use_cpu': [], 'listen': True, 'port': 7860, 'freeze': False, 'auth':
                         None, 'auth_file': None, 'autolaunch': False, 'docs': False, 'api_only': False, 'api_log': False, 'device_id':
                         None, 'cors_origins': None, 'cors_regex': None, 'tls_keyfile': None, 'tls_certfile': None, 'tls_selfsign': None,
                         'server_name': None, 'no_hashing': False, 'no_download': False, 'profile': False, 'disable_queue': False,
                         'subpath': None, 'backend': None, 'f': False, 'ui_settings_file': 'W:\\outputs\\config.json', 'ui_config_file':
                         'W:\\outputs\\ui-config.json', 'hide_ui_dir_config': False, 'theme': None, 'disable_console_progressbars': True,
                         'disable_safe_unpickle': True, 'lowram': False, 'disable_extension_access': False, 'api': True, 'api_auth': None,
                         'debug': True, 'reset': False, 'upgrade': True, 'use_ipex': False, 'use_directml': False, 'use_cuda': False,
                         'use_rocm': False, 'skip_update': False, 'skip_requirements': False, 'skip_extensions': False, 'skip_git': False,
                         'skip_torch': False, 'experimental': False, 'reinstall': False, 'test': False, 'version': False, 'ignore': False,
                         'safe': False, 'base': False, 'ldsr_models_path': 'W:\\models\\LDSR', 'lora_dir': 'W:\\models\\Lora',
                         'scunet_models_path': 'W:\\models\\ScuNET', 'swinir_models_path': 'W:\\models\\SwinIR', 'lyco_dir':
                         'W:\\models\\LyCORIS', 'lyco_patch_lora': False, 'lyco_debug': False, 'controlnet_dir': None,
                         'controlnet_annotator_models_path': None, 'no_half_controlnet': None, 'controlnet_preprocessor_cache_size': 16,
                         'controlnet_loglevel': 'INFO', 'ad_no_huggingface': False}
16:49:08-747125 INFO     Server arguments: ['--data-dir', 'W:\\outputs', '--models-dir', 'W:\\models', '--insecure', '--listen',
                         '--upgrade', '--debug']
16:49:08-766142 DEBUG    Memory used: 0.34 total: 63.91 Collected 0
Press any key to continue . . .

There does not appear to be anything else in sdnext.log beyond that.

vladmandic commented 1 year ago

That "press any key..." is weird, no idea where that is coming from and that is blocking launch to proceed. I'll go over the code to match with your log tomorrow.

vladmandic commented 1 year ago

install finished (some errors, but should not be critical), but actual server never even started executing. i've added some additional debug messages, please update and rerun (with --debug)

example of correct sequence, starting from Server arguments which is pretty much last thing you see:

09:22:57-660875 INFO     Server arguments: ['--debug', '--ckpt', 'models/v1-5-pruned-emaonly.safetensors']
09:22:57-666434 DEBUG    Memory used: 0.43 total: 27.41 Collected 0
09:22:57-667034 DEBUG    Loading module: ModuleSpec(name='webui', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f0675e09c00>, origin='/home/vlado/dev/automatic/webui.py')
09:22:57-667735 DEBUG    Starting module: <module 'webui' from '/home/vlado/dev/automatic/webui.py'>
09:22:57-670373 DEBUG    Loading Torch
09:22:59-794348 DEBUG    Loading Gradio
09:23:00-239244 DEBUG    Loading Modules
09:23:01-049687 INFO     Libraries loaded
09:23:01-051640 DEBUG    Entering start sequence
inktomi commented 1 year ago

Here's the output.. when I "press any key" webui.bat exits and I'm back at the command prompt.

PS G:\automatic> .\webui.bat --data-dir W:\outputs --models-dir W:\models --insecure --listen --upgrade --debug
Using VENV: G:\automatic\venv
07:55:27-709742 INFO     Starting SD.Next
07:55:27-722259 INFO     Python 3.10.11 on Windows
07:55:27-758318 INFO     Version: ec6d0548 Fri Jun 9 09:23:32 2023 -0400
07:55:28-314722 INFO     Setting environment tuning
07:55:28-315723 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False
07:55:28-316724 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True
07:55:28-318726 INFO     nVidia CUDA toolkit detected
07:55:38-323248 INFO     Torch 2.0.1+cu118
07:55:38-363825 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8700
07:55:38-365827 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 23028 Arch (8, 9) Cores 128
07:55:38-525062 DEBUG    Repository update time: Fri Jun  9 06:23:32 2023
07:55:38-526063 INFO     Verifying requirements
07:55:38-541083 INFO     Verifying packages
07:55:38-542084 INFO     Installing repositories
07:55:41-343888 INFO     Installing submodules
07:55:42-529711 DEBUG    Submodules list: 123d1da15d802823480f8020312ce449523f10e2
                         extensions-builtin/a1111-sd-webui-lycoris (heads/main)
                          9e6bbd9b8931bbe869a8e28e7005b0e13c2efff0 extensions-builtin/clip-interrogator-ext (heads/main)
                          70b3c5ea3c9f684d04e7ff59167565974415735c
                         extensions-builtin/multidiffusion-upscaler-for-automatic1111 (sd-webui-v1.1.0-25-g70b3c5e)
                          f02cacfc923e8bbf73f25327d722d50c458d66bb extensions-builtin/sd-dynamic-thresholding
                         (heads/master)
                          064c856acaed8c73c3a232e39df8c37d0bb9a15c extensions-builtin/sd-extension-system-info
                         (heads/main)
                          45dfe5977ea43a14f61f46d572615efe169ca2be extensions-builtin/sd-webui-agent-scheduler
                         (heads/main)
                         +2598ca9e78f167014c6556cb89dab792a745749f extensions-builtin/sd-webui-controlnet (heads/main)
                          b2f6e4cbe507078e3bf2b9d5fce6f83e2e233aa8
                         extensions-builtin/stable-diffusion-webui-images-browser (heads/main)
                          3d9eedbbf0d585207f97d5b21e42f32c0042df70 extensions-builtin/stable-diffusion-webui-rembg
                         (heads/master)
                          6417f5d7c183eccd79422f28804f7d7c507e05b9 modules/lora (v0.6.4-104-g6417f5d)
                          57d71913bf47a48dc7dd519b89dd30e09c937b83 modules/lycoris (heads/main)
                         +801130e4c45c815c4b2718a70ae88a2653fd780e wiki (heads/master)
07:55:43-228971 INFO     Updating submodules
07:55:50-468168 DEBUG    Installed packages: 241
07:55:50-469168 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding',
                         'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
07:55:51-896431 DEBUG    Running extension installer: G:\automatic\extensions-builtin\clip-interrogator-ext\install.py
07:56:29-769151 ERROR    Error running extension installer:
                         G:\automatic\extensions-builtin\clip-interrogator-ext\install.py
07:56:29-770152 DEBUG
07:56:35-111390 DEBUG    Running extension installer:
                         G:\automatic\extensions-builtin\sd-extension-system-info\install.py
07:56:36-504438 DEBUG    Running extension installer:
                         G:\automatic\extensions-builtin\sd-webui-agent-scheduler\install.py
07:56:37-899276 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-webui-controlnet\install.py
07:56:40-099845 DEBUG    Running extension installer: G:\automatic\extensions-builtin\seed_travel\install.py
07:56:41-451374 DEBUG    Running extension installer:
                         G:\automatic\extensions-builtin\stable-diffusion-webui-images-browser\install.py
07:56:44-923153 DEBUG    Running extension installer:
                         G:\automatic\extensions-builtin\stable-diffusion-webui-rembg\install.py
07:56:46-056961 DEBUG    Extensions all: ['adetailer', 'sd-webui-deforum']
07:56:46-832947 DEBUG    Running extension installer: W:\outputs\extensions\adetailer\install.py
07:56:48-651900 DEBUG    Running extension installer: W:\outputs\extensions\sd-webui-deforum\install.py
07:56:49-502945 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding',
                         'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'adetailer',
                         'sd-webui-deforum']
07:56:49-503947 INFO     Updating Wiki
07:56:50-084765 WARNING  Setup complete with errors: 1
07:56:50-085766 WARNING  See log file for more details: G:\automatic\sdnext.log
07:56:50-127307 INFO     Extension preload: 0.0s G:\automatic\extensions-builtin
07:56:50-138317 INFO     Extension preload: 0.0s W:\outputs\extensions
07:56:50-139318 INFO     Server arguments: ['--data-dir', 'W:\\outputs', '--models-dir', 'W:\\models', '--insecure',
                         '--listen', '--upgrade', '--debug']
07:56:50-179354 DEBUG    Memory used: 0.34 total: 63.91 Collected 0
07:56:50-180355 DEBUG    Loading module: ModuleSpec(name='webui', loader=<_frozen_importlib_external.SourceFileLoader
                         object at 0x000001B0F57A7B80>, origin='G:\\automatic\\webui.py')
07:56:50-181356 DEBUG    Starting module: <module 'webui' from 'G:\\automatic\\webui.py'>
07:56:50-198371 DEBUG    Loading Torch
Press any key to continue . . .
vladmandic commented 1 year ago

that is when torch ecosystem is getting loaded: torch, torchvision, pytorch_lightning and somehow that causes process to simply exit. that tells me there is something really wrong with torch installation. and you have rtx4090, so torch should be rock-solid. log also shows its the correct version of torch, but its non-functional anyhow.

i'd delete venv folder and let installer reinstall everything.

inktomi commented 1 year ago

I'll give that a try. When I activated the venv manually and ran the install script for the clip interrogator I saw a brief popup window before the script exited - no errors. I'm not sure if that's related though, as that script errors much earlier in the process than where the "Press any key.." is printed. I'll try deleting the venv folder today. Thank you for the ideas!

vladmandic commented 1 year ago

clip interrogator is important and we should look why it failed to install, but we're not even getting to the point where its used, lets focus on torch first.

inktomi commented 1 year ago

Here are the logs having deleted the venv folder. Still no luck.

PS G:\automatic> .\webui.bat --data-dir W:\outputs --models-dir W:\models --insecure --listen --upgrade --debug
Creating venv in directory G:\automatic\venv using python "C:\Users\....\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe"
Using VENV: G:\automatic\venv
09:10:35-021625 INFO     Starting SD.Next
09:10:35-034637 INFO     Python 3.10.11 on Windows
09:10:35-083682 INFO     Version: ec6d0548 Fri Jun 9 09:23:32 2023 -0400
09:10:35-113211 INFO     Setting environment tuning
09:10:35-114213 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False
09:10:35-115214 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True
09:10:35-117215 INFO     nVidia CUDA toolkit detected
09:10:35-118216 DEBUG    Package version not found: torch
09:10:35-118216 DEBUG    Package version not found: torchvision
09:10:35-119217 INFO     Installing package: torch torchvision --index-url https://download.pytorch.org/whl/cu118
09:10:35-120218 DEBUG    Running pip: install --upgrade torch torchvision --index-url https://download.pytorch.org/whl/cu118
09:12:15-232402 INFO     Torch 2.0.1+cu118
09:12:15-270437 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8700
09:12:15-271438 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 23028 Arch (8, 9) Cores 128
09:12:15-272439 DEBUG    Package version not found: tensorflow
09:12:15-273440 INFO     Installing package: tensorflow==2.12.0
09:12:15-274441 DEBUG    Running pip: install --upgrade tensorflow==2.12.0
09:12:39-997215 DEBUG    Repository update time: Fri Jun  9 06:23:32 2023
09:12:39-997717 INFO     Verifying requirements
09:12:39-998719 DEBUG    Package version not found: addict
09:12:39-999720 INFO     Installing package: addict
09:12:40-000721 DEBUG    Running pip: install --upgrade addict
09:12:40-813740 DEBUG    Package version not found: aenum
09:12:40-814742 INFO     Installing package: aenum
09:12:40-815742 DEBUG    Running pip: install --upgrade aenum
09:12:41-688807 DEBUG    Package version not found: aiohttp
09:12:41-689808 INFO     Installing package: aiohttp
09:12:41-689808 DEBUG    Running pip: install --upgrade aiohttp
09:12:43-640153 DEBUG    Package version not found: anyio
09:12:43-641154 INFO     Installing package: anyio
09:12:43-642155 DEBUG    Running pip: install --upgrade anyio
09:12:44-909086 DEBUG    Package version not found: appdirs
09:12:44-910087 INFO     Installing package: appdirs
09:12:44-911088 DEBUG    Running pip: install --upgrade appdirs
09:12:45-712087 DEBUG    Package version not found: astunparse
09:12:45-713089 INFO     Installing package: astunparse
09:12:45-714089 DEBUG    Running pip: install --upgrade astunparse
09:12:46-525093 DEBUG    Package version not found: bitsandbytes
09:12:46-526093 INFO     Installing package: bitsandbytes
09:12:46-527094 DEBUG    Running pip: install --upgrade bitsandbytes
09:12:48-969878 DEBUG    Package version not found: blendmodes
09:12:48-970880 INFO     Installing package: blendmodes
09:12:48-971880 DEBUG    Running pip: install --upgrade blendmodes
09:12:49-852447 DEBUG    Package version not found: clean-fid
09:12:49-853448 INFO     Installing package: clean-fid
09:12:49-854448 DEBUG    Running pip: install --upgrade clean-fid
09:12:58-470567 DEBUG    Package version not found: easydev
09:12:58-471568 INFO     Installing package: easydev
09:12:58-472568 DEBUG    Running pip: install --upgrade easydev
09:12:59-931922 DEBUG    Package version not found: extcolors
09:12:59-932923 INFO     Installing package: extcolors
09:12:59-933925 DEBUG    Running pip: install --upgrade extcolors
09:13:01-007173 DEBUG    Package version not found: facexlib
09:13:01-008174 INFO     Installing package: facexlib
09:13:01-009175 DEBUG    Running pip: install --upgrade facexlib
09:13:15-478883 DEBUG    Package version not found: filetype
09:13:15-479885 INFO     Installing package: filetype
09:13:15-480886 DEBUG    Running pip: install --upgrade filetype
09:13:16-602684 DEBUG    Package version not found: font-roboto
09:13:16-603685 INFO     Installing package: font-roboto
09:13:16-604686 DEBUG    Running pip: install --upgrade font-roboto
09:13:17-634403 DEBUG    Package version not found: fonts
09:13:17-635404 INFO     Installing package: fonts
09:13:17-636405 DEBUG    Running pip: install --upgrade fonts
09:13:18-606062 DEBUG    Package version not found: future
09:13:18-607063 INFO     Installing package: future
09:13:18-608063 DEBUG    Running pip: install --upgrade future
09:13:20-730561 DEBUG    Package version not found: gdown
09:13:20-731562 INFO     Installing package: gdown
09:13:20-732563 DEBUG    Running pip: install --upgrade gdown
09:13:22-164392 DEBUG    Package version not found: gfpgan
09:13:22-165393 INFO     Installing package: gfpgan
09:13:22-166394 DEBUG    Running pip: install --upgrade gfpgan
09:13:34-354142 DEBUG    Package version not found: GitPython
09:13:34-355143 INFO     Installing package: GitPython
09:13:34-356144 DEBUG    Running pip: install --upgrade GitPython
09:13:36-044718 DEBUG    Package version not found: httpcore
09:13:36-045719 INFO     Installing package: httpcore
09:13:36-045719 DEBUG    Running pip: install --upgrade httpcore
09:13:37-498325 DEBUG    Package version not found: inflection
09:13:37-499326 INFO     Installing package: inflection
09:13:37-500327 DEBUG    Running pip: install --upgrade inflection
09:13:38-581092 DEBUG    Package version not found: jsonmerge
09:13:38-582094 INFO     Installing package: jsonmerge
09:13:38-583093 DEBUG    Running pip: install --upgrade jsonmerge
09:13:40-152200 DEBUG    Package version not found: kornia
09:13:40-153201 INFO     Installing package: kornia
09:13:40-154202 DEBUG    Running pip: install --upgrade kornia
09:13:43-305469 DEBUG    Package version not found: lark
09:13:43-306470 INFO     Installing package: lark
09:13:43-307471 DEBUG    Running pip: install --upgrade lark
09:13:44-617498 DEBUG    Package version not found: lmdb
09:13:44-618499 INFO     Installing package: lmdb
09:13:44-619500 DEBUG    Running pip: install --upgrade lmdb
09:13:45-693255 DEBUG    Package version not found: lpips
09:13:45-694257 INFO     Installing package: lpips
09:13:45-695258 DEBUG    Running pip: install --upgrade lpips
09:13:46-827116 DEBUG    Package version not found: omegaconf
09:13:46-828118 INFO     Installing package: omegaconf
09:13:46-829118 DEBUG    Running pip: install --upgrade omegaconf
09:13:48-580321 DEBUG    Package version not found: open-clip-torch
09:13:48-581322 INFO     Installing package: open-clip-torch
09:13:48-582322 DEBUG    Running pip: install --upgrade open-clip-torch
09:13:54-372207 DEBUG    Package version not found: opencv-contrib-python
09:13:54-373208 INFO     Installing package: opencv-contrib-python
09:13:54-373208 DEBUG    Running pip: install --upgrade opencv-contrib-python
09:13:56-474186 DEBUG    Package version not found: piexif
09:13:56-475187 INFO     Installing package: piexif
09:13:56-476188 DEBUG    Running pip: install --upgrade piexif
09:13:57-778150 DEBUG    Package version not found: Pillow
09:13:57-779150 INFO     Installing package: Pillow
09:13:57-780151 DEBUG    Running pip: install --upgrade Pillow
09:14:00-017724 DEBUG    Package version not found: psutil
09:14:00-018726 INFO     Installing package: psutil
09:14:00-019726 DEBUG    Running pip: install --upgrade psutil
09:14:01-594708 DEBUG    Package version not found: pyyaml
09:14:01-595708 INFO     Installing package: pyyaml
09:14:01-596709 DEBUG    Running pip: install --upgrade pyyaml
09:14:02-814091 DEBUG    Package version not found: realesrgan
09:14:02-814091 INFO     Installing package: realesrgan
09:14:02-815092 DEBUG    Running pip: install --upgrade realesrgan
09:14:04-255695 DEBUG    Package version not found: resize-right
09:14:04-256696 INFO     Installing package: resize-right
09:14:04-257697 DEBUG    Running pip: install --upgrade resize-right
09:14:05-506887 DEBUG    Package version not found: rich
09:14:05-507888 INFO     Installing package: rich
09:14:05-508889 DEBUG    Running pip: install --upgrade rich
09:14:06-766810 DEBUG    Package version not found: safetensors
09:14:06-767810 INFO     Installing package: safetensors
09:14:06-767810 DEBUG    Running pip: install --upgrade safetensors
09:14:08-014219 DEBUG    Package version not found: scipy
09:14:08-015220 INFO     Installing package: scipy
09:14:08-016221 DEBUG    Running pip: install --upgrade scipy
09:14:09-286166 DEBUG    Package version not found: tb_nightly
09:14:09-287167 INFO     Installing package: tb_nightly
09:14:09-287167 DEBUG    Running pip: install --upgrade tb_nightly
09:14:10-814580 DEBUG    Package version not found: toml
09:14:10-815581 INFO     Installing package: toml
09:14:10-816582 DEBUG    Running pip: install --upgrade toml
09:14:12-082511 DEBUG    Package version not found: torchdiffeq
09:14:12-083511 INFO     Installing package: torchdiffeq
09:14:12-083511 DEBUG    Running pip: install --upgrade torchdiffeq
09:14:13-440780 DEBUG    Package version not found: torchsde
09:14:13-441781 INFO     Installing package: torchsde
09:14:13-442782 DEBUG    Running pip: install --upgrade torchsde
09:14:15-195670 DEBUG    Package version not found: voluptuous
09:14:15-196670 INFO     Installing package: voluptuous
09:14:15-197671 DEBUG    Running pip: install --upgrade voluptuous
09:14:16-489895 DEBUG    Package version not found: yapf
09:14:16-490895 INFO     Installing package: yapf
09:14:16-491896 DEBUG    Running pip: install --upgrade yapf
09:14:17-721308 DEBUG    Package version not found: scikit-image
09:14:17-722308 INFO     Installing package: scikit-image
09:14:17-723309 DEBUG    Running pip: install --upgrade scikit-image
09:14:18-982238 DEBUG    Package version not found: basicsr
09:14:18-983234 INFO     Installing package: basicsr
09:14:18-984235 DEBUG    Running pip: install --upgrade basicsr
09:14:20-287204 DEBUG    Package version not found: compel
09:14:20-288205 INFO     Installing package: compel
09:14:20-289206 DEBUG    Running pip: install --upgrade compel
09:14:35-535336 DEBUG    Package version not found: requests
09:14:35-536336 INFO     Installing package: requests==2.31.0
09:14:35-537337 DEBUG    Running pip: install --upgrade requests==2.31.0
09:14:37-341579 DEBUG    Package version not found: tqdm
09:14:37-342509 INFO     Installing package: tqdm==4.65.0
09:14:37-343510 DEBUG    Running pip: install --upgrade tqdm==4.65.0
09:14:38-969515 DEBUG    Package version not found: accelerate
09:14:38-970515 INFO     Installing package: accelerate==0.18.0
09:14:38-971516 DEBUG    Running pip: install --upgrade accelerate==0.18.0
09:14:41-116014 DEBUG    Package version not found: opencv-python
09:14:41-117015 INFO     Installing package: opencv-python==4.7.0.72
09:14:41-118016 DEBUG    Running pip: install --upgrade opencv-python==4.7.0.72
09:14:42-767166 DEBUG    Package version not found: diffusers
09:14:42-768166 INFO     Installing package: diffusers==0.16.1
09:14:42-769168 DEBUG    Running pip: install --upgrade diffusers==0.16.1
09:14:46-244666 DEBUG    Package version not found: einops
09:14:46-245667 INFO     Installing package: einops==0.4.1
09:14:46-246668 DEBUG    Running pip: install --upgrade einops==0.4.1
09:14:47-945256 DEBUG    Package version not found: gradio
09:14:47-946256 INFO     Installing package: gradio==3.32.0
09:14:47-946256 DEBUG    Running pip: install --upgrade gradio==3.32.0
09:15:05-976557 DEBUG    Package version not found: numexpr
09:15:05-977558 INFO     Installing package: numexpr==2.8.4
09:15:05-978559 DEBUG    Running pip: install --upgrade numexpr==2.8.4
09:15:07-956905 DEBUG    Package version not found: numpy
09:15:07-957906 INFO     Installing package: numpy==1.23.5
09:15:07-958907 DEBUG    Running pip: install --upgrade numpy==1.23.5
09:15:09-860678 DEBUG    Package version not found: numba
09:15:09-861679 INFO     Installing package: numba==0.57.0
09:15:09-861679 DEBUG    Running pip: install --upgrade numba==0.57.0
09:15:11-670855 DEBUG    Package version not found: pandas
09:15:11-671856 INFO     Installing package: pandas==1.5.3
09:15:11-672857 DEBUG    Running pip: install --upgrade pandas==1.5.3
09:15:25-423851 DEBUG    Package version not found: protobuf
09:15:25-424852 INFO     Installing package: protobuf==3.20.3
09:15:25-425853 DEBUG    Running pip: install --upgrade protobuf==3.20.3
09:15:27-267330 DEBUG    Package version not found: pytorch_lightning
09:15:27-269332 INFO     Installing package: pytorch_lightning==1.9.4
09:15:27-269332 DEBUG    Running pip: install --upgrade pytorch_lightning==1.9.4
09:15:32-482709 DEBUG    Package version not found: transformers
09:15:32-483710 INFO     Installing package: transformers==4.26.1
09:15:32-484711 DEBUG    Running pip: install --upgrade transformers==4.26.1
09:15:46-826367 DEBUG    Package version not found: timm
09:15:46-826367 INFO     Installing package: timm==0.6.13
09:15:46-827367 DEBUG    Running pip: install --upgrade timm==0.6.13
09:15:50-509070 DEBUG    Package version not found: tomesd
09:15:50-510071 INFO     Installing package: tomesd==0.1.2
09:15:50-511072 DEBUG    Running pip: install --upgrade tomesd==0.1.2
09:15:52-373811 DEBUG    Package version not found: urllib3
09:15:52-374808 INFO     Installing package: urllib3==1.26.15
09:15:52-375809 DEBUG    Running pip: install --upgrade urllib3==1.26.15
09:15:54-549338 INFO     Verifying packages
09:15:54-550340 DEBUG    Package version not found: clip
09:15:54-551341 INFO     Installing package: git+https://github.com/openai/CLIP.git
09:15:54-551341 DEBUG    Running pip: install --upgrade git+https://github.com/openai/CLIP.git
09:16:00-711085 DEBUG    Package version not found: onnxruntime
09:16:00-712086 INFO     Installing package: onnxruntime==1.14.0
09:16:00-713087 DEBUG    Running pip: install --upgrade onnxruntime==1.14.0
09:16:04-680789 INFO     Installing repositories
09:16:07-519204 INFO     Installing submodules
09:16:08-658049 DEBUG    Submodules list: 123d1da15d802823480f8020312ce449523f10e2 extensions-builtin/a1111-sd-webui-lycoris (heads/main)
                          9e6bbd9b8931bbe869a8e28e7005b0e13c2efff0 extensions-builtin/clip-interrogator-ext (heads/main)
                          70b3c5ea3c9f684d04e7ff59167565974415735c extensions-builtin/multidiffusion-upscaler-for-automatic1111 (sd-webui-v1.1.0-25-g70b3c5e)
                          f02cacfc923e8bbf73f25327d722d50c458d66bb extensions-builtin/sd-dynamic-thresholding (heads/master)
                          064c856acaed8c73c3a232e39df8c37d0bb9a15c extensions-builtin/sd-extension-system-info (heads/main)
                          45dfe5977ea43a14f61f46d572615efe169ca2be extensions-builtin/sd-webui-agent-scheduler (heads/main)
                          05e669696d119aa4a8e228a3743a773e363fabee extensions-builtin/sd-webui-controlnet (heads/main)
                          b2f6e4cbe507078e3bf2b9d5fce6f83e2e233aa8 extensions-builtin/stable-diffusion-webui-images-browser (heads/main)
                          3d9eedbbf0d585207f97d5b21e42f32c0042df70 extensions-builtin/stable-diffusion-webui-rembg (heads/master)
                          6417f5d7c183eccd79422f28804f7d7c507e05b9 modules/lora (v0.6.4-104-g6417f5d)
                          57d71913bf47a48dc7dd519b89dd30e09c937b83 modules/lycoris (heads/main)
                          646540f19208a22077f3814a20025c05b54b6699 wiki (heads/master)
09:16:09-176811 INFO     Updating submodules
09:16:16-031997 DEBUG    Installed packages: 179
09:16:16-032998 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'ScuNET',
                         'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
09:16:17-285922 DEBUG    Running extension installer: G:\automatic\extensions-builtin\clip-interrogator-ext\install.py
09:16:22-226365 INFO     Extension installed packages: clip-interrogator-ext ['clip-interrogator==0.5.4', 'timm==0.4.12', 'fairscale==0.4.4', 'blip-ci==0.0.3']
09:16:25-953108 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-extension-system-info\install.py
09:16:27-073141 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-webui-agent-scheduler\install.py
09:16:32-211202 INFO     Extension installed packages: sd-webui-agent-scheduler ['SQLAlchemy==2.0.15', 'greenlet==2.0.2']
09:16:32-761221 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-webui-controlnet\install.py
09:16:49-759408 INFO     Extension installed packages: sd-webui-controlnet ['pywin32==306', 'lxml==4.9.2', 'reportlab==4.0.4', 'pycparser==2.21', 'portalocker==2.7.0', 'cffi==1.15.1',
                         'svglib==1.5.1', 'tinycss2==1.2.1', 'mediapipe==0.10.1', 'tabulate==0.9.0', 'cssselect2==0.7.0', 'webencodings==0.5.1', 'sounddevice==0.4.6', 'iopath==0.1.9',
                         'yacs==0.1.8', 'fvcore==0.1.5.post20221221']
09:16:51-015333 DEBUG    Running extension installer: G:\automatic\extensions-builtin\seed_travel\install.py
09:16:54-018921 INFO     Extension installed packages: seed_travel ['imageio-ffmpeg==0.4.8']
09:16:54-594702 DEBUG    Running extension installer: G:\automatic\extensions-builtin\stable-diffusion-webui-images-browser\install.py
09:16:58-265877 INFO     Extension installed packages: stable-diffusion-webui-images-browser ['Send2Trash==1.8.2', 'image-reward==1.3']
09:16:58-801634 DEBUG    Running extension installer: G:\automatic\extensions-builtin\stable-diffusion-webui-rembg\install.py
09:17:04-912524 INFO     Extension installed packages: stable-diffusion-webui-rembg ['platformdirs==3.5.1', 'rembg==2.0.38', 'pooch==1.7.0', 'PyMatting==1.1.8']
09:17:05-052155 DEBUG    Extensions all: ['adetailer', 'sd-webui-deforum']
09:17:05-871171 DEBUG    Running extension installer: W:\outputs\extensions\adetailer\install.py
09:17:10-052103 INFO     Extension installed packages: adetailer ['ultralytics==8.0.115', 'seaborn==0.12.2']
09:17:10-941693 DEBUG    Running extension installer: W:\outputs\extensions\sd-webui-deforum\install.py
09:17:16-450657 INFO     Extension installed packages: sd-webui-deforum ['av==10.0.0', 'slicerator==1.1.0', 'PIMS==0.6.0']
09:17:16-451658 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'ScuNET',
                         'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'adetailer',
                         'sd-webui-deforum']
09:17:16-453660 INFO     Updating Wiki
09:17:17-029462 DEBUG    Setup complete without errors: 1686327437
09:17:17-039471 INFO     Extension preload: 0.0s G:\automatic\extensions-builtin
09:17:17-050481 INFO     Extension preload: 0.0s W:\outputs\extensions
09:17:17-051482 INFO     Server arguments: ['--data-dir', 'W:\\outputs', '--models-dir', 'W:\\models', '--insecure', '--listen', '--upgrade', '--debug']
09:17:17-090518 DEBUG    Memory used: 0.33 total: 63.91 Collected 0
09:17:17-091518 DEBUG    Loading module: ModuleSpec(name='webui', loader=<_frozen_importlib_external.SourceFileLoader object at 0x000001BD3FF68E80>, origin='G:\\automatic\\webui.py')
09:17:17-092520 DEBUG    Starting module: <module 'webui' from 'G:\\automatic\\webui.py'>
09:17:17-098526 DEBUG    Loading Torch
Press any key to continue . . .

That did make \clip-interrogator-ext\install.py succeed, so that's a step.

vladmandic commented 1 year ago

i've never seen anything like this before. i wrote a short test script that pretty much does what server does when it fails:

D:\SDNext>venv\Scripts\activate
python

from rich import print
import torch
import torchvision
import pytorch_lightning
import torch.nn.functional as F
print(torch.__version__)
2.0.1+cu118
print(torchvision.__version__)
0.15.2+cu118
print(pytorch_lightning.__version__)
1.9.4
x = torch.tensor([[1.5,.0,.0,.0]]).to('cuda').half()
layerNorm = torch.nn.LayerNorm(4, eps=0.00001, elementwise_affine=True, dtype=torch.float16, device='cuda')
y = layerNorm(x)
print(y)
tensor([[ 1.7324, -0.5771, -0.5771, -0.5771]], device='cuda:0',
       dtype=torch.float16, grad_fn=<NativeLayerNormBackward0>)
inktomi commented 1 year ago

I wish I knew more python (or Windows dev..) to be able to help... I'm happy to run scripts or anything you need to collect more info. With the updates pushed today:

PS G:\automatic> .\webui.bat --data-dir W:\outputs --models-dir W:\models --insecure --listen --upgrade --debug
Using VENV: G:\automatic\venv
19:24:04-749634 INFO     Starting SD.Next
19:24:04-761147 INFO     Python 3.10.11 on Windows
19:24:04-800182 INFO     Version: ec6d0548 Fri Jun 9 09:23:32 2023 -0400
19:24:05-337185 INFO     Updating main repository
19:24:08-028949 INFO     Upgraded to version: 971e51fe Fri Jun 9 18:24:18 2023 -0400
19:24:08-029949 INFO     Setting environment tuning
19:24:08-030950 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False
19:24:08-031951 DEBUG    Torch allowed: cuda=True rocm=True ipex=True diml=True
19:24:08-033953 INFO     nVidia CUDA toolkit detected
19:24:36-036792 INFO     Torch 2.0.1+cu118
19:24:36-060814 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8700
19:24:36-062816 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 23028 Arch (8, 9) Cores 128
19:24:36-063817 DEBUG    Package version not found: tensorflow
19:24:36-064819 INFO     Installing package: tensorflow==2.12.0
19:24:36-065820 DEBUG    Running pip: install --upgrade tensorflow==2.12.0
19:25:11-629969 INFO     Verifying requirements
19:25:11-644982 WARNING  Package wrong version: timm 0.4.12 required 0.6.13
19:25:11-645983 INFO     Installing package: timm==0.6.13
19:25:11-646984 DEBUG    Running pip: install --upgrade timm==0.6.13
19:25:15-448299 INFO     Verifying packages
19:25:15-450300 INFO     Installing repositories
19:25:18-247677 INFO     Installing submodules
19:25:18-872516 ERROR    Error running git: None / submodule
19:25:18-873517 DEBUG    Git output: 123d1da15d802823480f8020312ce449523f10e2 extensions-builtin/a1111-sd-webui-lycoris (heads/main)
                          9e6bbd9b8931bbe869a8e28e7005b0e13c2efff0 extensions-builtin/clip-interrogator-ext (heads/main)
                         +70b3c5ea3c9f684d04e7ff59167565974415735c extensions-builtin/multidiffusion-upscaler-for-automatic1111 (sd-webui-v1.1.0-25-g70b3c5e)
                          f02cacfc923e8bbf73f25327d722d50c458d66bb extensions-builtin/sd-dynamic-thresholding (heads/master)

                         fatal: no submodule mapping found in .gitmodules for path 'extensions-builtin/sd-extension-aesthetic-scorer'
19:25:18-875519 DEBUG    Submodules list: 123d1da15d802823480f8020312ce449523f10e2 extensions-builtin/a1111-sd-webui-lycoris (heads/main)
                          9e6bbd9b8931bbe869a8e28e7005b0e13c2efff0 extensions-builtin/clip-interrogator-ext (heads/main)
                         +70b3c5ea3c9f684d04e7ff59167565974415735c extensions-builtin/multidiffusion-upscaler-for-automatic1111 (sd-webui-v1.1.0-25-g70b3c5e)
                          f02cacfc923e8bbf73f25327d722d50c458d66bb extensions-builtin/sd-dynamic-thresholding (heads/master)

                         fatal: no submodule mapping found in .gitmodules for path 'extensions-builtin/sd-extension-aesthetic-scorer'
19:25:18-877520 WARNING  Attempting repository recover
19:25:21-467450 INFO     Continuing setup
19:25:22-124821 INFO     Updating submodules
19:25:29-046590 DEBUG    Installed packages: 222
19:25:29-047591 DEBUG    Extensions all: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'ScuNET',
                         'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
19:25:30-377580 DEBUG    Running extension installer: G:\automatic\extensions-builtin\clip-interrogator-ext\install.py
19:25:49-660914 ERROR    Error running extension installer: G:\automatic\extensions-builtin\clip-interrogator-ext\install.py
19:25:49-662915 DEBUG
19:25:54-588229 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-extension-system-info\install.py
19:25:55-903198 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-webui-agent-scheduler\install.py
19:25:57-219412 DEBUG    Running extension installer: G:\automatic\extensions-builtin\sd-webui-controlnet\install.py
19:25:59-355957 DEBUG    Running extension installer: G:\automatic\extensions-builtin\seed_travel\install.py
19:26:00-681995 DEBUG    Running extension installer: G:\automatic\extensions-builtin\stable-diffusion-webui-images-browser\install.py
19:26:02-821087 DEBUG    Running extension installer: G:\automatic\extensions-builtin\stable-diffusion-webui-rembg\install.py
19:26:03-853853 DEBUG    Extensions all: ['adetailer', 'sd-webui-deforum']
19:26:04-661904 DEBUG    Running extension installer: W:\outputs\extensions\adetailer\install.py
19:26:06-386508 DEBUG    Running extension installer: W:\outputs\extensions\sd-webui-deforum\install.py
19:26:07-176489 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora', 'multidiffusion-upscaler-for-automatic1111', 'ScuNET',
                         'sd-dynamic-thresholding', 'sd-extension-aesthetic-scorer', 'sd-extension-steps-animation', 'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sd-webui-controlnet', 'sd-webui-model-converter', 'seed_travel', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR', 'adetailer',
                         'sd-webui-deforum']
19:26:07-178491 INFO     Updating Wiki
19:26:07-828101 WARNING  Setup complete with errors: 2
19:26:07-829102 WARNING  See log file for more details: G:\automatic\sdnext.log
19:26:07-865639 INFO     Extension preload: 0.0s G:\automatic\extensions-builtin
19:26:07-876649 INFO     Extension preload: 0.0s W:\outputs\extensions
19:26:07-877650 INFO     Server arguments: ['--data-dir', 'W:\\outputs', '--models-dir', 'W:\\models', '--insecure', '--listen', '--upgrade', '--debug']
19:26:07-917686 DEBUG    Memory used: 0.34 total: 63.91 Collected 0
19:26:07-918687 DEBUG    Loading module: ModuleSpec(name='webui', loader=<_frozen_importlib_external.SourceFileLoader object at 0x000001ED5BEC7C40>, origin='G:\\automatic\\webui.py')
19:26:07-919688 DEBUG    Starting module: <module 'webui' from 'G:\\automatic\\webui.py'>
19:26:07-935703 DEBUG    Loading Torch
Press any key to continue . . .
vladmandic commented 1 year ago

just run code i pasted, line by line

inktomi commented 1 year ago

I saw that torch supports up to Python 3.11 now with 2.0.1 (https://github.com/pytorch/pytorch/issues/86566) so I tried that as well, but tat doesn't work as the overall app requires 3.10

inktomi commented 1 year ago

I had to change it slightly due to reported syntax errors..

from rich import print
import torch
import torchvision
import pytorch_lightning
import torch.nn.functional as F
print(torch.__version__)
#2.0.1+cu118
print(torchvision.__version__)
#0.15.2+cu118
print(pytorch_lightning.__version__)
#1.9.4
x = torch.tensor([[1.5,.0,.0,.0]]).to('cuda').half()
layerNorm = torch.nn.LayerNorm(4, eps=0.00001, elementwise_affine=True, dtype=torch.float16, device='cuda')
y = layerNorm(x)
print(y)
tensor([[ 1.7324, -0.5771, -0.5771, -0.5771]], device='cuda:0',dtype=torch.float16, grad_fn=NativeLayerNormBackward0)

When I activate the virtual env and then run it using python test.py there's no output:

(venv) PS G:\automatic> python test.py
Traceback (most recent call last):
  File "G:\automatic\test.py", line 2, in <module>
    import torch
  File "G:\automatic\venv\lib\site-packages\torch\__init__.py", line 117, in <module>
    res = kernel32.LoadLibraryExW(dll, None, 0x00001100)
KeyboardInterrupt

(I eventually ctrl+c'd it)

Python version 3.10.11

vladmandic commented 1 year ago

thats why i asked to run it line by line in interactive python and not as a full script - so you can see which line locks it up.

inktomi commented 1 year ago

🤦‍♂️ I'm sorry!

import torch takes a while, but works. import torchvision drops me out of the interactive python shell and back to the command prompt.

(venv) PS G:\automatic> python
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from rich import print
>>> import torch
>>> import torchvision
(venv) PS G:\automatic>

I also ran with python -v to help debug why the import may be failing, but nothing stands out other than perhaps something in the torchvision c bindings?

python-v.txt

vladmandic commented 1 year ago

now we know what is failing, but still have no idea why. normally i'd say torchvision install is bad or its not matching installed torch, but you've run install with fresh venv and there were no errors during torch and torchvision installation, so i'm pretty much at a loss. plus the python log doesn't actually show any errors, it just drops to prompt which means process was hard killed before it even managed to print an error.

you can experiment using torch and torchvision nightly builds to try different versions, but other than that, i think this needs to be reported to torch repo.

inktomi commented 1 year ago

I can do that - thank you for getting me this far on it! I appreciate it.

inktomi commented 1 year ago

Of all the random things that may have went sidways - reinstalling the Microsoft Visual C++ runtime seems to have solved this.

vladmandic commented 1 year ago

daaamn, i would not have expected that. if nothing else, i would expect that torchvision reports it if has a problem with that!