[Bug]: Lora/Lycoris list is stale when switching between SD base model versions

[zixaphir]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

Without the option Always show all networks on the Lora page when switching between model base versions such as SD1.5 to SDXL, the list of Lora/Lycoris will not be updated to account for the base model. Lora will be shown that are meant for SD1.5 while ones compatible with SDXL will be hidden. This is easy to work around by manually refreshing the list, but this creates confusion for end users when they are searching for a Lora and are not aware of this.

Steps to reproduce the problem

• Ensure the Always show all networks on the Lora page option is disabled or the issue is irrelevant as this option side-steps a stale list
• Ensure you have at least one Lora for each model base version that you are testing for, such as SD2, SD1.5, or SDXL, with the SD Version metadata correctly populated
• Open the Lora tab and let it load entirely
• Switch to a model of a different base version, such as from SDXL to SD1.5
• The Lora list will be stale and containing Lora for the incorrect SD version.

What should have happened?

The Lora list should contain Lora models of the correct SD version, either by making the incorrect models hidden and the correct ones visible, or refreshing entirely to show only the correct models, plus maybe unknown SD version models if the option to do so is enabled.

Sysinfo

https://gist.github.com/zixaphir/2a922fdf2cc12fc87cd778c039c33c47

What browsers do you use to access the UI ?

Mozilla Firefox, Google Chrome, Brave

Console logs

./webui.sh                                                                                                                                                                                                                           

################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye), Fedora 34+ and openSUSE Leap 15.4 or newer.
################################################################

################################################################
Running on **REDACTED** user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Accelerating launch.py...
################################################################
Using TCMalloc: libtcmalloc_minimal.so.4
2023-12-05 11:11:11.299914: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Python 3.11.6 (main, Nov 14 2023, 09:36:21) [GCC 13.2.1 20230801]
Version: v1.7.0-RC-5-gf92d6149
Commit hash: f92d61497a426a19818625c3ccdaae9beeb82b31
Launching Web UI with arguments: --xformers --enable-insecure-extension-access --medvram --listen
2023-12-05 11:11:15.472940: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Civitai Helper: Get Custom Model Folder
[-] ADetailer initialized. version: 23.11.1, num models: 13
2023-12-05 11:11:20,503 - ControlNet - INFO - ControlNet v1.1.419
ControlNet preprocessor location: /home/zixaphir/nvme/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads
2023-12-05 11:11:20,567 - ControlNet - INFO - ControlNet v1.1.419
Loading weights **REDACTED** from **REDACTED**
2023-12-05 11:11:20,695 - AnimateDiff - INFO - Injecting LCM to UI.
2023-12-05 11:11:20,914 - AnimateDiff - INFO - Hacking i2i-batch.
Civitai Helper: Set Proxy: 
Creating model from config: **REDACTED**/stable-diffusion-webui/configs/v1-inference.yaml
Loading VAE weights specified in settings: **REDACTED**/nvme/stable-diffusion-webui/models/VAE/vae-ft-mse-840000-ema-pruned.ckpt
Applying attention optimization: xformers... done.
Model loaded in 1.7s (load weights from disk: 0.5s, create model: 0.2s, apply weights to model: 0.2s, load VAE: 0.5s, calculate empty prompt: 0.2s).
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 12.0s (prepare environment: 1.3s, import torch: 3.1s, import gradio: 0.4s, setup paths: 0.6s, other imports: 0.3s, setup codeformer: 0.2s, list SD models: 0.6s, load scripts: 1.6s, create ui: 3.7s, gradio launch: 0.1s).
Reusing loaded model **REDACTED** to load **REDACTED**
Calculating sha256 for **REDACTED**: **REDACTED**
Loading weights **REDACTED** from **REDACTED**
Creating model from config: **REDACTED**/stable-diffusion-webui/repositories/generative-models/configs/inference/sd_xl_base.yaml
Loading VAE weights specified in settings: **REDACTED**/stable-diffusion-webui/models/VAE/vae-ft-mse-840000-ema-pruned.ckpt # This is the wrong VAE but it is because my test SDXL model has no metadata for VAE added yet
Applying attention optimization: xformers... done.
Model loaded in 2.0s (create model: 0.6s, apply weights to model: 1.0s, load VAE: 0.1s, calculate empty prompt: 0.2s).

Additional information

I am using a 1.7.0RC but it was only to ensure the issue still exists in recent dev versions. The issue was originally brought to my attention here: https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/issues/58

[w-e-w]

irrc there is actually more issues current behavior the Lora types that is displayed is based on what model is loaded in memory, whitch may is not nessery the same as the one in the settings

this can hpaaen when you switch model in during generation, like using refine or a diffrent checkpoint fo hires pass also if override setting is used to switch the model, the backend will have know Idea what should display after the generation is fnnished, the model will not be automatic loaded beck to the default (this is done to reduce unsessery model loading of models)

I think a rework of this entire system is required

my initial thought is to just add a menu of some type to let users filter / choose the model type

[zixaphir]

my initial thought is to just add a menu of some type to let users filter / choose the model type

Honestly from my perspective, better filtering in general would be nice, but that would definitely be good for this particular issue

[w-e-w]

in addition to the types, I like to also add something something like a tag field in the user metadata then the user can filter by tag like I only wish to see lora with character and favorites tag for SDXL the model type can be set as auto detect or manual

currently user can only organize one's model be directory