[Bug] Models in subdirectories do not work (should be fixed but waiting on confirmation)

[TFWol]

Could there be support for models in sub directories like below? EX:

- stable-diffusion-webui/
    - models/
        - Stable-diffusion/
            - aaaaa.ckpt
            - aaaaa.png
            - aaaaa.txt
            - models_to_test/   <---sub directories like these don't work
              - bbbbb.ckpt
              - bbbbb.png
              - bbbbb.txt
            - favorite_models/
              - ccccc.ckpt
              - ccccc.png
              - ccccc.txt

Similar to how the original project this is forked from allowed. It would make organizing a bit less burdensome with a large amount of files.

[CurtisDS]

I never removed support for that. It should still work the same way. I have all my preview files in a sub directory right now. Your setup should work.

What is the current behaviour you are seeing? Are the models from the subdirectory showing "No Preview Found"? Or are they missing from the drop down entirely? And are you using symlinks?

[TFWol]

When putting models and preview images in something likestable-diffusion-webui\models\Stable-diffusion\Good, refresh model list, then select the model it will usually show 'no preview found'. If I move them all to the base stable-diffusion-webui\models\Stable-diffusion\ folder it works.

For the times it errors, I'll reproduce it and provide an example.

[CurtisDS]

I just pushed an update, could you update your extension before testing anything. Just so I can narrow down the issue.

[TFWol]

After the update, I'm getting quite a few trace errors mentioning re.error: bad escape \m at position 9 I'll try setting up a separate stable diffusion instance to see if it persists

[TFWol]

hmmm, could you see if you get similar errors by making a subdirectory in 'stable-diffusion-webui/models/Stable-diffusion/'

so like:

stable-diffusion-webui/
  - models/
    - Stable-diffusion/
      - Test_directory/
        - your-model.ckpt
        - your-model.png

[CurtisDS]

I tried that and my model wouldnt show up in a1111.

this worked though

stable-diffusion-webui/
  - models/
    - Stable-diffusion/
      - your-model.ckpt
      - Test_directory/
        - your-model.png

I dont know why a1111 isnt loading checkpoints from sub directories. It works for hypernetworks and embeddings. I tried this and it worked:

stable-diffusion-webui/
  - models/
    - hypernetworks/
      - Previews/
        - your-model.pt
        - your-model.png

Im not getting any trace errors. What operating system are you using?

[TFWol]

That's odd that models aren't showing in sub-directories, but hypernet and embeddings are. I know that models show up when using the original code, maybe useful to see what's different?

Using the layout from base folder works, also when using a subdirectory for previews only like you mentioned.

I'm on Win10

[TFWol]

Here's the results I'm getting:

mdjrny-v4.safetensors [038351b1] from Huggingface

Scenario 1

Models and previews in base directory

stable-diffusion-webui/
  - models/
    - Stable-diffusion/
      - mdjrny-v4.safetensors
      - mdjrny-v4.png

Result: Works

Scenario 2

Models in base directory with previews in subdirectory

stable-diffusion-webui/
  - models/
    - Stable-diffusion/
      - mdjrny-v4.safetensors
      - Previews/
        - mdjrny-v4.png

Result: Works

Scenario 3

Models and previews in sub-directory

stable-diffusion-webui/
  - models/
    - Stable-diffusion/
      - Test_directory/
        - mdjrny-v4.safetensors
        - mdjrny-v4.png

Result: Error

Edit:

• fixed example names

[TFWol]

Prob not related, but I'm also seeing that the name matching might be a little too permissive:

[CurtisDS]

Ok I see whats happening... still not sure why though.

And its something I changed from the original that I didnt think was doing anything.

When you (~again I dont know why its working for you and not for me~ it was because I didnt realize the folder name was included in the name so it was sorted to the top of the list and I was looking at the bottom) put a checkpoint in a subdirectory it adds that subdirectory name to the model name shown in the drop down. Are you on an older version of a1111 maybe?

anyway, then the code takes the name and adds it to a regex string to match on. And because the model name has a backslash in it the regex thinks thats an escape character. Which is why it was complaining about \m. I can add back in the code from the original that should fix that issue.

Also in regards to the name matching being too permissive... right now (and in the original) the matching was basically {anything}{model name}{anything}.png. Obviously this can cause naming collisions (addressed here). But I also am working on a new update with a setting to turn on strict name matching that should address that issue.

Thanks for the help on narrowing down the issue. I should be able to post an update later today.

[TFWol]

You just beat me to the punch. I tried editing the net request by adding more escapes (\): old: {"fn_index":257,"data":["Test_directory\\amdjrny-v4.safetensors [038351b1]"],"session_hash":"5b250gomp7d"}

edit: {"fn_index":257,"data":["Test_directory\\\\amdjrny-v4.safetensors [038351b1]"],"session_hash":"5b250gomp7d"} (copied the model for testing) That made it return the right values in dev tools.

I'm using Auto1111 commit 82725f0ac439f7e3b67858d55900e95330bbd326 (Jan 13)

No problem helping; I didn't know if it was just me or something.

[CurtisDS]

Hopefully I got this sorted now, if you could update and see if it works

[TFWol]

No errors, but I wasn't able to get any directory to show me images this time around, even in the base directory. Same result when enabling strict naming and following the naming scheme (ex: modelName.1.png, modelName.preview.png)

I fully restarting stable diffusion and cleared browser cache between changes.

While I was at it, I checked hypernetworks and embeddings since I hadn't done so before. No issues there, including with sub-directories.

Edit:

• I'll try the bleeding edge version of Automatic1111 after I finish combing through the code diffs.

[TFWol]

Looks like a TON changed in 2 weeks between versions. At some point the way model paths work was overhauled: https://github.com/AUTOMATIC1111/stable-diffusion-webui/compare/82725f0...master#diff-fe2129c140fc5cff6a2b769eaffa44583ed8dc64e7aee39f52377e4624c7a84eR270

I'm able to see sub-directory stuff after switching to bleeding edge. I'll test everything else out and post back results.

[TFWol]

Everything is working, including with the strict naming settings, except for 1 specific model (so far): mdjrny-v4.safetensors , but only when it's in the base model directory.

• if I put it and all related files in stable-diffusion-webui\models\Stable-diffusionand reload it won't show anything (no errors)
• if I put them in a sub-directory and reload, it shows everything fine I have no clue how to explain that one 😶

btw, I really like the strict naming option.

[CurtisDS]

No errors, but I wasn't able to get any directory to show me images this time around, even in the base directory. Same result when enabling strict naming and following the naming scheme (ex: modelName.1.png, modelName.preview.png)

Damn... I can try installing the old version again and see if i can get it working with the old one. But it should have worked and things change so fast idk if trying to debug an old version is really worth it :(

Everything is working, including with the strict naming settings, except for 1 specific model (so far): mdjrny-v4.safetensors , but only when it's in the base model directory.

Thats very strange. You were using the exact same files? No chance of a typo? I just tried it with my own copy of mdjrny-v4.safetensors and a mdjrny-v4.png and it worked in the main stable-diffusion-webui\models\Stable-diffusion folder.

[CurtisDS]

I think I've solved it now 🙄. Sorry, I should have tested the 0d908078619110f35d8dcf6dd4917bddfd8cf306 commit more thoroughly before pushing it.

I wrote a regex pattern to extract the model name from the string which appears in the model drop down list (where this extension gets the list of models from). Annoyingly... a1111 handles the string in the drop down differently in different situations. The checkpoint drop down includes the file extension and it also used to always include the hash inside of square brackets. Now for whatever reason it only sometimes adds the hash code. And in the last commit when I switched to using a regex to extract the model name (before I was doing a more simple replace statement) I got the order of the extension and the hash wrong. My regex pattern was looking for MODEL [HASH].EXT when the patter is actually MODEL.EXT [HASH]. So I think the reason why sometimes it worked and sometimes it didn't was because the regex would not extract the model name properly if the hash was included but would when it wasn't. Which also explains why on the old version none of them worked... because it always included the hash in the drop down and so always failed to find the model name.

Programming sometimes is fun. lol.

[Aryetis]

Looks like it's still not working :/ It's not picking my .txt nor my pictures

Sidenote : It looks like there's a lot of work and bugs being done with the new "Extra Networks" feature : https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/7338 . Hopefuly they'll manage to make this feature read .jpg in subdirectories too and not just a .png right next to the model

[CurtisDS]

Looks like it's still not working :/ It's not picking my .txt nor my pictures

you need to rename your images and text file to follow the naming convention described here: Name Matching Rules

That should solve the issue for you.

[TFWol]

Sorry for the delay. Things got busy where I'm at. Looks good, I'm able to see images and text for models placed in sub-drectories and in base folder. Great work!

The only thing I see that seems odd, just as an fyi, is if you have a model in a subdirectory, the script will still search other unrelated directories in other subfolders and grabs images from them.

example:

stable-diffusion-webui/
  - models/
    - Stable-diffusion/
      - Test_directory/
        - mdjrny-v4.safetensors
        - mdjrny-v4.png
      - Another_directory
        - mdjrny-v4.png   <---- this would be picked up too 

Bad example that doesn't bother my workflow at, just something to keep in mind in case someone has a weird situation.

Programming sometimes is fun. lol.

Yes, yes it is XD