[Issue]: Huggingface models are not working on the dev branch

aaronsantiago commented 9 months ago

Issue Description

I have been using SDXL Turbo on main with no issue, but currently I can't use the model on dev. The version platform description below might show that it's using the original backend and not the diffusers backend, but I've tried it on both.

Version Platform Description

15:42:39-791625 INFO Starting SD.Next 15:42:39-793097 INFO Logger: file="/home/sensorium/workspace/auto-accel-dev/sdnext.log" level=INFO size=57262 mode=append 15:42:39-793769 INFO Python 3.11.4 on Linux 15:42:39-799692 INFO Version: app=sd.next updated=2024-02-06 hash=ad69dcc7 url=https://github.com/vladmandic/automatic/tree/dev 15:42:39-994353 INFO Latest published version: 9bf08c5493925b6ddb3451f6a6e00460ca92aedd 2024-01-30T14:29:51Z 15:42:39-997915 INFO Platform: arch=x86_64 cpu=x86_64 system=Linux release=6.5.0-15-generic python=3.11.4 15:42:40-001967 INFO nVidia CUDA toolkit detected: nvidia-smi present 15:42:40-035181 WARNING Modified files: ['stable_fast-1.0.3+torch212cu121-cp311-cp311-manylinux2014_x86_64.whl'] 15:42:40-041564 INFO Extensions: disabled=[] 15:42:40-043209 INFO Extensions: enabled=['stable-diffusion-webui-rembg', 'sd-webui-agent-scheduler', 'sd-extension-chainner', 'Lora', 'stable-diffusion-webui-images-browser', 'sd-extension-system-info', 'sd-webui-controlnet'] extensions-builtin 15:42:40-046796 INFO Extensions: enabled=[] extensions 15:42:40-049373 INFO Startup: quick launch 15:42:40-051322 INFO Verifying requirements 15:42:40-066759 INFO Verifying packages 15:42:40-067652 INFO Extensions: disabled=[] 15:42:40-068019 INFO Extensions: enabled=['stable-diffusion-webui-rembg', 'sd-webui-agent-scheduler', 'sd-extension-chainner', 'Lora', 'stable-diffusion-webui-images-browser', 'sd-extension-system-info', 'sd-webui-controlnet'] extensions-builtin 15:42:40-068628 INFO Extensions: enabled=[] extensions 15:42:40-069800 INFO Extension preload: {'extensions-builtin': 0.0, 'extensions': 0.0} 15:42:40-070575 INFO Command line args: ['--listen', '--docs', '--cors-origins', '', '--insecure'] insecure=True listen=True docs=True cors_origins= channel 3: open failed: connect failed: Connection refused 15:42:41-872538 INFO Load packages: torch=2.1.2+cu121 diffusers=0.26.1 gradio=3.43.2 15:42:42-101074 INFO Engine: backend=Backend.DIFFUSERS compute=cuda device=cuda attention="Scaled-Dot-Product" mode=no_grad 15:42:42-115988 INFO Device: device=NVIDIA GeForce RTX 4090 n=1 arch=sm_90 cap=(8, 9) cuda=12.1 cudnn=8902 driver=535.154.05 15:42:42-501171 INFO ONNX: selected=CUDAExecutionProvider, available=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'AzureExecutionProvider', 'CPUExecutionProvider'] 15:42:42-533981 INFO Available VAEs: path="models/VAE" items=0 15:42:42-534567 INFO Disabled extensions: ['sd-webui-controlnet'] 15:42:42-535161 INFO Available models: path="models/Stable-diffusion" items=1 time=0.00 15:42:42-672137 INFO LoRA networks: available=0 folders=2 15:42:42-799690 INFO Extension: script='extensions-builtin/sd-webui-agent-scheduler/scripts/task_scheduler.py' Using sqlite file: extensions-builtin/sd-webui-agent-scheduler/task_scheduler.sqlite3 15:42:42-895824 INFO Extensions init time: 0.28 sd-webui-agent-scheduler=0.12 stable-diffusion-webui-images-browser=0.09 15:42:43-017181 INFO Load UI theme: name="black-teal" style=Auto base=sdnext.css 15:42:43-403433 INFO Extension list is empty: refresh required 15:42:43-662238 INFO Local URL: http://localhost:7860/ 15:42:43-662746 INFO API Docs: http://localhost:7860/docs 15:42:43-663151 INFO API ReDocs: http://localhost:7860/redocs

Relevant log output

15:42:43-818420 ERROR    StableDiffusionPipeline: Error no file named pytorch_model.bin, tf_model.h5,
                         model.ckpt.index or flax_model.msgpack found in directory
                         models/Diffusers/models--stabilityai--sdxl-turbo/snapshots/f4b0486b498f84668e828044de
                         1d0c8ba486e05b/text_encoder.
15:42:43-819157 ERROR    Failed loading model:
                         models/Diffusers/models--stabilityai--sdxl-turbo/snapshots/f4b0486b498f84668e828044de
                         1d0c8ba486e05b auto=Error no file named pytorch_model.bin, tf_model.h5,
                         model.ckpt.index or flax_model.msgpack found in directory
                         models/Diffusers/models--stabilityai--sdxl-turbo/snapshots/f4b0486b498f84668e828044de
                         1d0c8ba486e05b/text_encoder. diffusion=Error no file named pytorch_model.bin,
                         tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory
                         models/Diffusers/models--stabilityai--sdxl-turbo/snapshots/f4b0486b498f84668e828044de
                         1d0c8ba486e05b/text_encoder.

Backend

Diffusers

Branch

Dev

Model

SD-XL

Acknowledgements

[X] I have read the above and searched for existing issues
[X] I confirm that this is classified correctly and its not an extension issue

vladmandic commented 9 months ago

Latest version of diffusers removed automatic fallback for loading model variant which is causing this issue. specifically, model downloaded is fp16 variant, but settings -> diffusers -> preferred model variant is set to default which in most cases means fp32.

so when you try to load a model, its basically saying there is no fp32 variant of the model available. (previously it would then load whichever variant IS available)

try setting preferred model variant to fp16 and load model then. even if that works, i'd consider it a workaround only and i need to figure out a better way to handle this in code.

vladmandic commented 9 months ago

Should be fixed in latest dev.

vladmandic / automatic