[Issue]: Models/Loras not loading with any method but server restart.

ComatoseHub commented 1 week ago

Issue Description

When a new model or download is added via hugging face or civitai it does not load into the UI until I fully restart the server.

Version Platform Description

16:14:25-553303 INFO Python version=3.10.11 platform=Windows bin="P:\SDNext\venv\Scripts\python.exe" venv="P:\SDNext\venv" 16:14:25-667095 INFO Version: app=sd.next updated=2024-09-13 hash=e7ec07f9 branch=master url=https://github.com/vladmandic/automatic/tree/master ui=main 16:14:26-290354 INFO Platform: arch=AMD64 cpu=AMD64 Family 25 Model 97 Stepping 2, AuthenticAMD system=Windows release=Windows-10-10.0.22631-SP0 python=3.10.11 16:14:26-291665 INFO HF cache folder: C:\Users[Username].cache\huggingface\hub 16:14:26-299260 INFO Python version=3.10.11 platform=Windows bin="P:\SDNext\venv\Scripts\python.exe" venv="P:\SDNext\venv" 16:14:46-596460 INFO Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg'] 16:14:46-925434 INFO Command line args: ['--use-zluda', '--upgrade', '--autolaunch'] autolaunch=True upgrade=True use_zluda=True 16:15:29-096666 INFO Load packages: {'torch': '2.3.0+cu118', 'diffusers': '0.31.0.dev0', 'gradio': '3.43.2'} 16:15:29-997412 INFO VRAM: Detected=11.98 GB Optimization=none 16:15:30-000448 INFO Engine: backend=Backend.DIFFUSERS compute=cuda device=cuda attention="Dynamic Attention BMM" mode=no_grad 16:15:30-010061 INFO Device: device=AMD Radeon RX 7700 XT [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver= 16:15:44-834609 INFO Browser session: user=None client=127.0.0.1 agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) Gecko/20100101 Firefox/131.0

Relevant log output

16:14:25-550223 INFO     Starting SD.Next
16:14:25-552249 INFO     Logger: file="P:\SDNext\sdnext.log" level=INFO size=13595 mode=append
16:14:25-553303 INFO     Python version=3.10.11 platform=Windows bin="P:\SDNext\venv\Scripts\python.exe" venv="P:\SDNext\venv"
16:14:25-667095 INFO     Version: app=sd.next updated=2024-09-13 hash=e7ec07f9 branch=master url=https://github.com/vladmandic/automatic/tree/master ui=main
16:14:26-290354 INFO     Platform: arch=AMD64 cpu=AMD64 Family 25 Model 97 Stepping 2, AuthenticAMD system=Windows release=Windows-10-10.0.22631-SP0 python=3.10.11
16:14:26-291665 INFO     HF cache folder: C:\Users\[Username]\.cache\huggingface\hub
16:14:26-299260 INFO     Python version=3.10.11 platform=Windows bin="P:\SDNext\venv\Scripts\python.exe" venv="P:\SDNext\venv"
16:14:26-302291 INFO     AMD ROCm toolkit detected
16:14:26-659308 INFO     ROCm agents detected: ['gfx1101']
16:14:26-660325 INFO     ROCm version detected: 6.1
16:14:26-661340 WARNING  ZLUDA support: experimental
16:14:27-692537 INFO     Using ZLUDA in P:\SDNext\.zluda
No ROCm runtime is found, using ROCM_HOME='C:\Program Files\AMD\ROCm\6.1'
16:14:39-811970 INFO     Verifying requirements
16:14:39-814123 INFO     Verifying packages
16:14:39-815153 INFO     Startup: standard
16:14:39-816168 INFO     Verifying submodules
16:14:46-596460 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg']
16:14:46-597496 INFO     Verifying requirements
16:14:46-598501 INFO     Updating Wiki
16:14:46-925434 INFO     Command line args: ['--use-zluda', '--upgrade', '--autolaunch'] autolaunch=True upgrade=True use_zluda=True
16:15:29-096666 INFO     Load packages: {'torch': '2.3.0+cu118', 'diffusers': '0.31.0.dev0', 'gradio': '3.43.2'}
16:15:29-997412 INFO     VRAM: Detected=11.98 GB Optimization=none
16:15:30-000448 INFO     Engine: backend=Backend.DIFFUSERS compute=cuda device=cuda attention="Dynamic Attention BMM" mode=no_grad
16:15:30-010061 INFO     Device: device=AMD Radeon RX 7700 XT [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver=
16:15:33-121635 INFO     Available VAEs: path="models\VAE" items=0
16:15:33-123172 INFO     Disabled extensions: ['sdnext-modernui']
16:15:33-125783 INFO     Available models: path="models\Stable-diffusion" items=2 time=0.00
16:15:33-375502 INFO     Extension: script='extensions-builtin\Lora\scripts\lora_script.py' [2;36m16:15:33-373441[0m[2;36m [0m[34mINFO    [0m LoRA networks: [33mavailable[0m=[1;36m2[0m [33mfolders[0m=[1;36m2[0m
16:15:35-069640 INFO     Extension: script='extensions-builtin\sd-webui-agent-scheduler\scripts\task_scheduler.py' Using sqlite file: extensions-builtin\sd-webui-agent-scheduler\task_scheduler.sqlite3
16:15:35-093200 INFO     UI theme: type=Standard name="invoked"
16:15:36-087644 INFO     Extension list is empty: refresh required
16:15:36-530968 INFO     Local URL: http://127.0.0.1:7860/
16:15:36-653354 INFO     [AgentScheduler] Task queue is empty
16:15:36-654377 INFO     [AgentScheduler] Registering APIs
16:15:36-750928 INFO     Setting Torch parameters: device=cuda dtype=torch.float16 vae=torch.float16 unet=torch.float16 context=no_grad fp16=True bf16=None optimization=Dynamic Attention BMM
16:15:36-753238 INFO     Startup time: 49.82 torch=25.43 gradio=16.52 diffusers=0.13 libraries=4.08 extensions=1.74 face-restore=0.20 ui-networks=0.11 ui-txt2img=0.05 ui-img2img=0.06 ui-control=0.10 ui-models=0.24 ui-settings=0.38 ui-extensions=0.32 launch=0.10 api=0.06
                         app-started=0.14
16:15:36-756569 INFO     Launching browser
16:15:44-834609 INFO     Browser session: user=None client=127.0.0.1 agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) Gecko/20100101 Firefox/131.0

Backend

Diffusers

UI

Standard

Branch

Master

Model

StableDiffusion 1.5

Acknowledgements

[X] I have read the above and searched for existing issues
[X] I confirm that this is classified correctly and its not an extension issue

vladmandic commented 1 week ago

add --debug to you command line params.
start sdnext
add a model to models\Stable-diffusion
press refresh button next to model list pulldown on top left
add a lora to models\Lora
press refresh button at the top of extra networks panel on the right side
upload full sdnext.log here
show content of that models folder and lora folder (dir command output)

ComatoseHub commented 1 week ago

sdnext.log Here is that log.

vladmandic commented 1 week ago

what type of storage/filesystem is "P:\" ? it seems its filesystem is not updating folder metadata on new file creation - so sdnext does not rescan folder because os said folder did not change.

ComatoseHub commented 1 week ago

OK seems for some reason that drive is formatted as exFAT and not NTFS let me try on an NTFS drive.

ComatoseHub commented 1 week ago

It appears the issue is resolved, ex fat drives on windows cause the issue.

vladmandic / automatic