[Bug]: Symlink subfolders no longer work in Extra Networks

[PaulCoddington]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

Symlink subfolders no longer load in Extra Networks. Issue #142 may have been prematurely closed.

Most recent commit where Extra Networks worked with symlinks:

Revision: https://github.com/anapnoe/stable-diffusion-webui-ux/commit/f2b9c2cb4fa5f0e866c1b6f84e44d12ff6653af3 Date: 12/05/2023 11:57:05 pm

Looks like symlinks were broken here:

Revision: https://github.com/anapnoe/stable-diffusion-webui-ux/commit/b08500cec8a791ef20082628b49b17df833f5dda Date: 13/05/2023 5:16:37 pm

Steps to reproduce the problem

1. Create a subfolder in any Extra Networks folder which links to another drive.

eg:

\models\LoRA\symlink\…

2. Refresh Extra Networks pane.

Symlink folder will be listed, but no models or subfolders within the symlink will be displayed. Standard subfolders are listed and contain models, as expected.

What should have happened?

Extra Networks pane should display models and subfolders contained within all subfolders, including symlink subfolders.

Commit where the problem happens

3c31e25

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Microsoft Edge

Command Line Arguments

--opt-sdp-attention
--opt-channelslast
--skip-install
--no-progressbar-hiding
--autolaunch
--ckpt-dir='%StableDiffusionWebUIRoot%\data\models\Stable-Diffusion'
--vae-dir='%StableDiffusionWebUIRoot%\data\models\VAE'
--styles-file='%StableDiffusionWebUIRoot%\data\models\Styles\styles.csv'
--embeddings-dir='%StableDiffusionWebUIRoot%\data\models\Embeddings'
--hypernetwork-dir='%StableDiffusionWebUIRoot%\data\models\HyperNetworks'
--lora-dir='%StableDiffusionWebUIRoot%\data\models\LoRA'
--lyco-dir='%StableDiffusionWebUIRoot%\data\models\LyCORIS'
--bsrgan-models-path='%StableDiffusionWebUIRoot%\data\models\BSRGAN'
--clip-models-path='%StableDiffusionWebUIRoot%\data\models\CLIP'
--controlnet-dir='%StableDiffusionWebUIRoot%\data\models\ControlNet'
--codeformer-models-path='%StableDiffusionWebUIRoot%\data\models\CodeFormer'
--esrgan-models-path='%StableDiffusionWebUIRoot%\data\models\ESRGAN'
--gfpgan-models-path='%StableDiffusionWebUIRoot%\data\models\GFPGAN'
--ldsr-models-path='%StableDiffusionWebUIRoot%\data\models\LDSR'
--realesrgan-models-path='%StableDiffusionWebUIRoot%\data\models\RealESRGAN'
--scunet-models-path='%StableDiffusionWebUIRoot%\data\models\ScuNET'
--swinir-models-path='%StableDiffusionWebUIRoot%\data\models\SwinIR'

List of extensions

a1111-sd-webui-lycoris a1111-sd-webui-tagcomplete clip-interrogator-ext multidiffusion-upscaler-for-automatic1111 sd-dynamic-thresholding sd-extension-aesthetic-scorer sd-extension-steps-animation sd-extension-system-info sd-webui-aspect-ratio-helper sd-webui-controlnet sd-webui-cutoff sd-webui-model-converter seed_travel stable-diffusion-webui-images-browser stable-diffusion-webui-rembg ultimate-upscale-for-automatic1111

Console logs

Python 3.10.6 | packaged by conda-forge | (main, Oct 24 2022, 16:02:16) [MSC v.1916 64 bit (AMD64)]
Version: <none>
Commit hash: 3c31e25f4e25a41003b0af074d107a5495408ffe
Launching Web UI with arguments: --opt-sdp-attention --opt-channelslast --skip-install --no-progressbar-hiding --autolaunch
No module 'xformers'. Proceeding without it.
Loading weights [6ce0161689] from C:\ProgramData\Stable Diffusion\data\models\Stable-Diffusion\[base]\Stable Diffusion.safetensors
Creating model from config: C:\ProgramData\Stable Diffusion\sd-webui-ux\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying scaled dot product cross attention optimization.
Textual inversion embeddings loaded(38): [redacted]
Model loaded in 4.0s (load weights from disk: 0.2s, create model: 0.4s, apply weights to model: 0.8s, apply channels_last: 0.4s, apply half(): 0.6s, move model to device: 0.5s, load textual inversion embeddings: 1.0s).
ControlNet v1.1.173
ControlNet v1.1.173
Image Browser: ImageReward is not installed, cannot be used.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 16.7s (import torch: 1.9s, import gradio: 1.5s, import ldm: 0.6s, other imports: 1.1s, setup codeformer: 0.1s, load scripts: 7.8s, create ui: 3.2s, gradio launch: 0.3s).

Additional information

No response

[PaulCoddington]

Amended local copy to resolve issue pending official fix.

modules/shared.py
- for root, dirs, files in os.walk(path):
+ for root, dirs, files in os.walk(path, followlinks=True):

modules/ui_extra_networks.py
- for root, dirs, files in os.walk(parentdir):
+ for root, dirs, files in os.walk(parentdir, followlinks=True):

Possibly relevant, but out of scope…

modules/textual_inversion/textual_inversion.py
- for root, dirs, fns in os.walk(shared.cmd_opts.textual_inversion_templates_dir):
+ for root, dirs, fns in os.walk(shared.cmd_opts.textual_inversion_templates_dir, followlinks=True):

Some 3rd party extensions may need similar amendments for some users depending on how they use symlinks (out of scope).

[anapnoe]

thanks @PaulCoddington