[Issue]: Torch cpu installed on nvidia system

SAC020 commented 1 week ago

Issue Description

start webui
all image generation parameters default (default sampler, cfg, steps, no refine, no loras etc)
entered a basic prompt, hit generate, nothing seems to happen (generation seems hung)

Noticed Torch generator: device=cpu in the log

Noticed there is some CPU load, and no GPU load

These are all the settings CPU-related I could find

Version Platform Description

06:25:11-946899 INFO Starting SD.Next 06:25:11-949892 INFO Logger: file="C:\ai\automatic\sdnext.log" level=DEBUG size=65 mode=create 06:25:11-951886 INFO Python: version=3.11.9 platform=Windows bin="C:\ai\automatic\venv\Scripts\python.exe" venv="C:\ai\automatic\venv" 06:25:12-156512 INFO Version: app=sd.next updated=2024-10-07 hash=072a8371 branch=dev url=https://github.com/vladmandic/automatic/tree/dev ui=dev 06:25:13-168581 INFO Repository latest available e7ec07f9783701629ca1411ad82aec87232501b9 2024-09-13T16:51:56Z 06:25:13-182544 INFO Platform: arch=AMD64 cpu=Intel64 Family 6 Model 165 Stepping 5, GenuineIntel system=Windows release=Windows-10-10.0.22631-SP0 python=3.11.9

Relevant log output

PS C:\ai\automatic> .\webui.bat --debug
Using VENV: C:\ai\automatic\venv
06:25:11-946899 INFO     Starting SD.Next
06:25:11-949892 INFO     Logger: file="C:\ai\automatic\sdnext.log" level=DEBUG size=65 mode=create
06:25:11-951886 INFO     Python: version=3.11.9 platform=Windows bin="C:\ai\automatic\venv\Scripts\python.exe"
                         venv="C:\ai\automatic\venv"
06:25:12-156512 INFO     Version: app=sd.next updated=2024-10-07 hash=072a8371 branch=dev
                         url=https://github.com/vladmandic/automatic/tree/dev ui=dev
06:25:13-168581 INFO     Repository latest available e7ec07f9783701629ca1411ad82aec87232501b9 2024-09-13T16:51:56Z
06:25:13-182544 INFO     Platform: arch=AMD64 cpu=Intel64 Family 6 Model 165 Stepping 5, GenuineIntel system=Windows
                         release=Windows-10-10.0.22631-SP0 python=3.11.9
06:25:13-184538 DEBUG    Setting environment tuning
06:25:13-185535 DEBUG    Torch allocator: "garbage_collection_threshold:0.80,max_split_size_mb:512"
06:25:13-199498 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False openvino=False zluda=False
06:25:13-210469 INFO     CUDA: nVidia toolkit detected
06:25:13-376661 WARNING  Modified files: ['models/Reference/playgroundai--playground-v2-1024px-aesthetic.jpg']
06:25:13-477391 INFO     Verifying requirements
06:25:13-482378 INFO     Verifying packages
06:25:13-533243 DEBUG    Timestamp repository update time: Mon Oct  7 18:48:00 2024
06:25:13-535237 INFO     Startup: standard
06:25:13-536234 INFO     Verifying submodules
06:25:17-016646 DEBUG    Git detached head detected: folder="extensions-builtin/sd-extension-chainner" reattach=main
06:25:17-017643 DEBUG    Git submodule: extensions-builtin/sd-extension-chainner / main
06:25:17-144304 DEBUG    Git detached head detected: folder="extensions-builtin/sd-extension-system-info" reattach=main
06:25:17-145331 DEBUG    Git submodule: extensions-builtin/sd-extension-system-info / main
06:25:17-279942 DEBUG    Git detached head detected: folder="extensions-builtin/sd-webui-agent-scheduler" reattach=main
06:25:17-280940 DEBUG    Git submodule: extensions-builtin/sd-webui-agent-scheduler / main
06:25:17-487198 DEBUG    Git detached head detected: folder="extensions-builtin/sdnext-modernui" reattach=dev
06:25:17-489192 DEBUG    Git submodule: extensions-builtin/sdnext-modernui / dev
06:25:17-661435 DEBUG    Git detached head detected: folder="extensions-builtin/stable-diffusion-webui-rembg"
                         reattach=master
06:25:17-662433 DEBUG    Git submodule: extensions-builtin/stable-diffusion-webui-rembg / master
06:25:17-795077 DEBUG    Git detached head detected: folder="modules/k-diffusion" reattach=master
06:25:17-796076 DEBUG    Git submodule: modules/k-diffusion / master
06:25:17-943680 DEBUG    Git detached head detected: folder="wiki" reattach=master
06:25:17-945676 DEBUG    Git submodule: wiki / master
06:25:18-170075 DEBUG    Register paths
06:25:18-287760 DEBUG    Installed packages: 221
06:25:18-289755 DEBUG    Extensions all: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sdnext-modernui',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg']
06:25:18-552052 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\sd-extension-system-info\install.py
06:25:18-789417 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\sd-webui-agent-scheduler\install.py
06:25:22-637552 DEBUG    Extension force: name=sd-webui-controlnet commit=ecd33eb
06:25:22-729293 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\sd-webui-controlnet\install.py
06:25:36-848580 DEBUG    Extension installer:
                         C:\ai\automatic\extensions-builtin\stable-diffusion-webui-images-browser\install.py
06:25:39-445909 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\stable-diffusion-webui-rembg\install.py
06:25:48-611646 DEBUG    Extensions all: []
06:25:48-612645 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sdnext-modernui',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg']
06:25:48-614639 INFO     Verifying requirements
06:25:48-615637 DEBUG    Setup complete without errors: 1728357949
06:25:48-625960 DEBUG    Extension preload: {'extensions-builtin': 0.01, 'extensions': 0.0}
06:25:48-627956 DEBUG    Starting module: <module 'webui' from 'C:\\ai\\automatic\\webui.py'>
06:25:48-628954 INFO     Command line args: ['--debug'] debug=True
06:25:48-629951 DEBUG    Env flags: []
06:26:03-085287 INFO     System packages: {'torch': '2.4.1+cpu', 'diffusers': '0.31.0.dev0', 'gradio': '3.43.2',
                         'transformers': '4.45.1', 'accelerate': '0.33.0'}
06:26:04-179518 DEBUG    Huggingface cache: folder="C:\Users\sebas\.cache\huggingface\hub"
06:26:04-215602 DEBUG    Read: file="config.json" json=36 bytes=1598 time=0.001
06:26:04-216626 DEBUG    Unknown settings: ['cross_attention_options', 'face_restoration_unload']
06:26:04-218593 INFO     Engine: backend=Backend.DIFFUSERS compute=None device=cpu attention="Scaled-Dot-Product"
                         mode=no_grad
06:26:04-221584 DEBUG    Read: file="html\reference.json" json=52 bytes=29118 time=0.001
06:26:04-264142 INFO     Torch parameters: backend=cpu device=cpu config=BF16 dtype=torch.bfloat16 vae=torch.bfloat16
                         unet=torch.bfloat16 context=no_grad nohalf=False nohalfvae=False upscast=False
                         deterministic=False test-fp16=True test-bf16=True optimization="Scaled-Dot-Product"
06:26:05-162788 DEBUG    ONNX: version=1.19.2 provider=CPUExecutionProvider, available=['AzureExecutionProvider',
                         'CPUExecutionProvider']
06:26:05-463331 INFO     Device:
06:26:05-618959 DEBUG    Importing LDM
06:26:05-658440 DEBUG    Entering start sequence
06:26:05-662907 DEBUG    Initializing
06:26:05-748554 INFO     Available VAEs: path="models\VAE" items=0
06:26:05-749552 INFO     Available UNets: path="models\UNET" items=0
06:26:05-750548 INFO     Available TEs: path="models\Text-encoder" items=4
06:26:05-752543 INFO     Disabled extensions: ['sd-webui-controlnet', 'sdnext-modernui']
06:26:05-759524 DEBUG    Read: file="cache.json" json=2 bytes=11134 time=0.001
06:26:05-771493 DEBUG    Read: file="metadata.json" json=708 bytes=2521081 time=0.010
06:26:05-810928 DEBUG    Scanning diffusers cache: folder="models\Diffusers" items=3 time=0.00
06:26:05-812377 INFO     Available Models: path="models\Stable-diffusion" items=15 time=0.06
06:26:05-997955 INFO     Available Yolo: path="models\yolo items=6 downloaded=1
06:26:05-998951 DEBUG    Load extensions
06:26:06-170231 INFO     Extension: script='extensions-builtin\Lora\scripts\lora_script.py'
                         [2;36m06:26:06-163250[0m[2;36m [0m[34mINFO    [0m Available LoRAs: [33mitems[0m=[1;36m143[0m
                         [33mfolders[0m=[1;36m6[0m
06:26:07-102979 INFO     Extension: script='extensions-builtin\sd-webui-agent-scheduler\scripts\task_scheduler.py' Using
                         sqlite file: extensions-builtin\sd-webui-agent-scheduler\task_scheduler.sqlite3
06:26:07-393344 DEBUG    Extensions init time: 1.39 Lora=0.05 sd-extension-chainner=0.10 sd-webui-agent-scheduler=0.81
                         stable-diffusion-webui-images-browser=0.28
06:26:07-435627 DEBUG    Read: file="html/upscalers.json" json=4 bytes=2672 time=0.000
06:26:07-437937 DEBUG    Read: file="extensions-builtin\sd-extension-chainner\models.json" json=24 bytes=2719 time=0.001
06:26:07-442534 DEBUG    chaiNNer models: path="models\chaiNNer" defined=24 discovered=0 downloaded=8
06:26:07-444661 DEBUG    Upscaler type=ESRGAN folder="models\ESRGAN" model="1x-ITF-SkinDiffDetail-Lite-v1"
                         path="models\ESRGAN\1x-ITF-SkinDiffDetail-Lite-v1.pth"
06:26:07-445657 DEBUG    Upscaler type=ESRGAN folder="models\ESRGAN" model="4xNMKDSuperscale_4xNMKDSuperscale"
                         path="models\ESRGAN\4xNMKDSuperscale_4xNMKDSuperscale.pth"
06:26:07-447653 DEBUG    Upscaler type=ESRGAN folder="models\ESRGAN" model="4x_NMKD-Siax_200k"
                         path="models\ESRGAN\4x_NMKD-Siax_200k.pth"
06:26:07-451642 INFO     Available Upscalers: items=56 downloaded=11 user=3 time=0.06 types=['None', 'Lanczos',
                         'Nearest', 'ChaiNNer', 'AuraSR', 'ESRGAN', 'LDSR', 'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
06:26:07-472585 INFO     Available Styles: folder="models\styles" items=288 time=0.02
06:26:07-481564 DEBUG    Creating UI
06:26:07-483557 DEBUG    UI themes available: type=Standard themes=12
06:26:07-484554 INFO     UI theme: type=Standard name="black-teal"
06:26:07-495573 DEBUG    UI theme: css="C:\ai\automatic\javascript\black-teal.css" base="sdnext.css" user="None"
06:26:07-499659 DEBUG    UI initialize: txt2img
06:26:07-584431 DEBUG    Networks: page='model' items=66 subfolders=2 tab=txt2img folders=['models\\Stable-diffusion',
                         'models\\Diffusers', 'models\\Reference'] list=0.05 thumb=0.01 desc=0.00 info=0.00 workers=8
                         sort=Default
06:26:07-599391 DEBUG    Networks: page='lora' items=143 subfolders=0 tab=txt2img folders=['models\\Lora',
                         'models\\LyCORIS'] list=0.06 thumb=0.02 desc=0.06 info=0.04 workers=8 sort=Default
06:26:07-631335 DEBUG    Networks: page='style' items=288 subfolders=1 tab=txt2img folders=['models\\styles', 'html']
                         list=0.05 thumb=0.00 desc=0.00 info=0.00 workers=8 sort=Default
06:26:07-636435 DEBUG    Networks: page='embedding' items=13 subfolders=0 tab=txt2img folders=['models\\embeddings']
                         list=0.04 thumb=0.01 desc=0.01 info=0.00 workers=8 sort=Default
06:26:07-639424 DEBUG    Networks: page='vae' items=0 subfolders=0 tab=txt2img folders=['models\\VAE'] list=0.00
                         thumb=0.00 desc=0.00 info=0.00 workers=8 sort=Default
06:26:07-903560 DEBUG    UI initialize: img2img
06:26:08-025538 DEBUG    UI initialize: control models=models\control
06:26:08-352512 DEBUG    Read: file="ui-config.json" json=0 bytes=2 time=0.001
06:26:08-464215 DEBUG    UI themes available: type=Standard themes=12
06:26:09-071619 DEBUG    Reading failed: C:\ai\automatic\html\extensions.json [Errno 2] No such file or directory:
                         'C:\\ai\\automatic\\html\\extensions.json'
06:26:09-073613 INFO     Extension list is empty: refresh required
06:26:09-703683 DEBUG    Extension list: processed=8 installed=8 enabled=6 disabled=2 visible=8 hidden=0
06:26:10-066468 DEBUG    Root paths: ['C:\\ai\\automatic']
06:26:10-175783 INFO     Local URL: http://127.0.0.1:7860/
06:26:10-176652 DEBUG    Gradio functions: registered=2490
06:26:10-180645 DEBUG    FastAPI middleware: ['Middleware', 'Middleware']
06:26:10-183636 DEBUG    Creating API
06:26:10-360193 INFO     [AgentScheduler] Task queue is empty
06:26:10-361162 INFO     [AgentScheduler] Registering APIs
06:26:10-692156 DEBUG    Scripts setup: ['IP Adapters:0.026', 'XYZ Grid:0.014', 'Face:0.014', 'AnimateDiff:0.007',
                         'CogVideoX:0.008', 'Ctrl-X:0.006', 'Prompt matrix:0.006', 'X/Y/Z Grid:0.014',
                         'Image-to-Video:0.007']
06:26:10-693768 DEBUG    Model metadata: file="metadata.json" no changes
06:26:10-694766 DEBUG    Model requested: fn=<lambda>
06:26:10-695764 INFO     Load model: select="epicrealismXL_v8Kiss [04639d4084]"
06:26:10-697759 DEBUG    Load model: target="C:\ai\automatic\models\Stable-diffusion\epicrealismXL_v8Kiss.safetensors"
                         existing=False info=None
06:26:10-698757 DEBUG    Load model: path="C:\ai\automatic\models\Stable-diffusion\epicrealismXL_v8Kiss.safetensors"
06:26:10-699754 INFO     Autodetect model: detect="Stable Diffusion XL" class=StableDiffusionXLPipeline
                         file="C:\ai\automatic\models\Stable-diffusion\epicrealismXL_v8Kiss.safetensors" size=6617MB
Diffusers  2.52s/it ████████ 100% 7/7 00:17 00:00 Loading pipeline components...
06:26:29-235061 DEBUG    Setting model: pipeline=StableDiffusionXLPipeline config={'low_cpu_mem_usage': True,
                         'torch_dtype': torch.bfloat16, 'load_connected_pipeline': True, 'extract_ema': False, 'config':
                         'configs/sdxl', 'use_safetensors': True, 'cache_dir':
                         'C:\\Users\\sebas\\.cache\\huggingface\\hub'}
06:26:34-898734 INFO     Load network: type=embeddings loaded=2 skipped=11 time=5.66
06:26:34-900728 DEBUG    Setting model: component=VAE slicing=True
06:26:34-901726 DEBUG    Setting model: attention="Scaled-Dot-Product"
06:26:34-925662 DEBUG    Setting model: offload=none
06:26:34-927657 DEBUG    Read: file="C:\ai\automatic\configs\sdxl\vae\config.json" json=15 bytes=674 time=0.000
06:26:35-169013 DEBUG    GC: utilization={'gpu': 0, 'ram': 11, 'threshold': 80} gc={'collected': 2435, 'saved': 0}
                         before={'gpu': 0, 'ram': 7.29} after={'gpu': 0, 'ram': 7.29, 'retries': 0, 'oom': 0} device=cpu
                         fn=load_diffuser time=0.24
06:26:35-177994 INFO     Load model: time=24.23 load=18.54 embeddings=5.66 native=1024 memory={'ram': {'used': 7.29,
                         'total': 63.92}}
06:26:35-180546 DEBUG    Script callback init time: image_browser.py:ui_tabs=0.44 system-info.py:app_started=0.07
                         task_scheduler.py:app_started=0.35
06:26:35-181543 INFO     Startup time: 46.54 torch=9.60 gradio=2.72 diffusers=1.39 libraries=3.27 samplers=0.09
                         extensions=1.39 models=0.06 detailer=0.19 upscalers=0.06 ui-networks=0.43 ui-txt2img=0.08
                         ui-img2img=0.08 ui-control=0.14 ui-settings=0.26 ui-extensions=1.11 ui-defaults=0.29
                         launch=0.17 api=0.09 app-started=0.42 checkpoint=24.49
06:26:35-183537 DEBUG    Save: file="config.json" json=36 bytes=1549 time=0.004
06:26:35-186530 DEBUG    Unused settings: ['cross_attention_options', 'face_restoration_unload']
06:27:11-157428 INFO     MOTD: N/A
06:27:14-905756 DEBUG    UI themes available: type=Standard themes=12
06:27:15-164115 INFO     Browser session: user=None client=127.0.0.1 agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64)
                         AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0
06:27:38-321280 INFO     Base: class=StableDiffusionXLPipeline
06:27:38-323276 DEBUG    Sampler: sampler=default class=EulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'beta_start': 0.00085, 'beta_end': 0.012, 'beta_schedule': 'scaled_linear', 'trained_betas':
                         None, 'prediction_type': 'epsilon', 'interpolation_type': 'linear', 'use_karras_sigmas': False,
                         'use_exponential_sigmas': False, 'use_beta_sigmas': False, 'sigma_min': None, 'sigma_max':
                         None, 'timestep_spacing': 'leading', 'timestep_type': 'discrete', 'steps_offset': 1,
                         'rescale_betas_zero_snr': False, 'final_sigmas_type': 'zero', 'clip_sample': False,
                         'sample_max_value': 1.0, 'set_alpha_to_one': False, 'skip_prk_steps': True}
06:27:38-351201 DEBUG    Torch generator: device=cpu seeds=[472781593]
06:27:38-352198 DEBUG    Diffuser pipeline: StableDiffusionXLPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1
                         set={'prompt': 1, 'negative_prompt': 1, 'guidance_scale': 6, 'num_inference_steps': 20, 'eta':
                         1.0, 'guidance_rescale': 0.7, 'denoising_end': None, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
06:28:00-274230 DEBUG    Server: alive=True jobs=1 requests=73 uptime=116 memory=7.24/63.92 backend=Backend.DIFFUSERS
                         state=idle
Progress ?it/s                                              0% 0/20 00:00 ? Base06:30:00-336096 DEBUG    Server: alive=True jobs=1 requests=331 uptime=236 memory=7.48/63.92 backend=Backend.DIFFUSERS
                         state=idle
06:32:00-381047 DEBUG    Server: alive=True jobs=1 requests=588 uptime=356 memory=7.47/63.92 backend=Backend.DIFFUSERS
                         state=idle
06:34:00-427052 DEBUG    Server: alive=True jobs=1 requests=845 uptime=476 memory=7.49/63.92 backend=Backend.DIFFUSERS
                         state=idle
06:36:00-472131 DEBUG    Server: alive=True jobs=1 requests=1103 uptime=596 memory=7.38/63.92 backend=Backend.DIFFUSERS
                         state=idle

Backend

Diffusers

UI

Standard

Branch

Dev

Model

StableDiffusion XL

Acknowledgements

[X] I have read the above and searched for existing issues
[X] I confirm that this is classified correctly and its not an extension issue

vladmandic commented 1 week ago

torch type cannot be changed in settings as it requires different torch. but yes, you can see in log that its really torch-cpu that is installed.

06:26:03-085287 INFO System packages: {'torch': '2.4.1+cpu', 'diffusers': '0.31.0.dev0', 'gradio': '3.43.2', 'transformers': '4.45.1', 'accelerate': '0.33.0'}

i don't know why it was installed, but i suggest to do a reinstall using:

webui --debug --reinstall

SAC020 commented 1 week ago

Thank you, reinstall did the trick and it's now working normally.

18:02:59-594146 INFO     System packages: {'torch': '2.4.1+cu124', 'diffusers': '0.31.0.dev0', 'gradio': '3.43.2',
                         'transformers': '4.45.1', 'accelerate': '0.33.0'}

...

18:04:38-818134 DEBUG    Torch generator: device=cuda seeds=[2245576255]

What I can tell from my side is that:

for sure I have not installed torch cpu on purpose
I did a git pull yesterday, and the previous one was sometime before the weekend, but I can't remember exactly when

Thanks again

vladmandic commented 1 week ago

sdnext installer attempts to autodetect gpu and install correct torch. sometimes autodetect can fail - in nvdia case, if nvidia-smi returns invalid result.

you can always force installation of specific torch variant by using --use-xxx flag on first sdnext startup. (e.g. --use-cuda, --use-rocm, --use-directml`, etc...)

vladmandic / automatic