search

pkgxdev / pantry

pkg manifests
https://pkgx.dev/pkgs/
172 stars 119 forks source link

Stable Diffusion WebUI improvements #2248

Open magnusviri opened 1 year ago

magnusviri commented 1 year ago

Over the weekend I started using stable-diffusion-webui (sdw) for more than just giggles and here is some feedback on how the tea distribution of sdw can improve. Currently, tea's sdw does this.

I've had to go into ~/.tea/stable-diffusion-webui/v*/lib/ to make many changes. Here's my suggested changes based on what I've had to do.

Change 1

The reason for this is because more things need to be linked to user space. See below.

Change 2

The reason for the above changes is because the models directory is extremely fluid. There are sdw features that download additional models. Here's a list of the directories that sdw has installed for me based on things I've done.

Here is an example of a LyCORIS model I had to install manually: john-berkey-style-lycoris

Also, I needed to find the Lora directory to install models there manually. Example: mecha

Change 3

The reason for this change is because there are tons of embeddings that can be installed here. Example: https://civitai.com/models/69768/marblingti

Optional change 4

Ideally, sdw can run with the stable-diffusion-webui folder read-only. I haven't really started working on this, but might some day. With that goal, here are other files in it's directory that change when running sdw that could be linked to other locations. I listed this as optional because this might be a major pain.

~/.tea/stable-diffusion-webui/v*/lib/cache.json
~/.tea/stable-diffusion-webui/v*/lib/cache.json.lock
~/.tea/stable-diffusion-webui/v*/lib/config.json
~/.tea/stable-diffusion-webui/v*/lib/interogate
~/.tea/stable-diffusion-webui/v*/lib/detected_maps
~/.tea/stable-diffusion-webui/v*/lib/params.txt
~/.tea/stable-diffusion-webui/v*/lib/tmp
~/.tea/stable-diffusion-webui/v*/lib/styles.csv
~/.tea/stable-diffusion-webui/v*/lib/styles.csv.bak

I don't know what created the interrogate folder. I think it was CLIP or DeepBooru interrogation. Either way, it kind of says to me that it will be futile to try to make the stable-diffusion-webui folder read-only. And of course, I didn't include any pycache directories. What a pain.

jhheider commented 1 year ago

Question: when you say "link $x to $y" do you mean ln -s $x $y (impossible at install the way tea currently works, unless we were to write a startup shim that creates those as needed), or do you mean ln -s $y $x, shipping with a bunch of probably broken symlinks? The latter could work but will the folders get created appropriately? The former would generally be corrected with runtime environment variables or a startup script (like whisper.cpp).

mxcl commented 1 year ago

Change link ~/.tea/stable-diffusion-webui/v*/lib/extensions to ~/.local/share/stable-diffusion-webui/extensions

yeah this is what was intended. that it is wrong is a bug. whoops!

link ~/.local/share/stable-diffusion-webui/models/Stable-diffusion to ~/.local/share/models/stable-diffusion

we do this.

link ~/.tea/stable-diffusion-webui/v*/lib/models to ~/.local/share/stable-diffusion-webui/models

can do though sd-webui puts a bunch of stuff in there itself on install so we'll be moving pkg data to userland which we try not to do. Might be a way around that…

~/.tea/stable-diffusion-webui/v*/lib/embeddings to ~/.local/share/stable-diffusion-webui/embeddings

nps

Ideally, sdw can run with the stable-diffusion-webui folder read-only

agreed. We may have to submit patches tho.

mxcl commented 1 year ago

unless we were to write a startup shim that creates those as needed

yep we have a startup shim that does this already.

mxcl commented 1 year ago

I note that sd-webui has a extensions and extensions-builtin folders so you can drop extensions in and not interfere with the builtins. We need to get them to do the same for models.

mxcl commented 1 year ago

link ~/.tea/stable-diffusion-webui/v*/lib/models to ~/.local/share/stable-diffusion-webui/models

can do though sd-webui puts a bunch of stuff in there itself on install so we'll be moving pkg data to userland which we try not to do. Might be a way around that…

actually there’s only:

./VAE-approx/model.pt
./karlo/ViT-L-14_stats.th

i'm not sure if these are actual models or not

jhheider commented 1 year ago

yep we have a startup shim that does this already.

I should just check that before I key up... 🤦

magnusviri commented 1 year ago

Sorry it's taken me so long to get back to this. I am debating if I should try to fix this and do a PR.

SDXL has been released and I think it might be better than 1.5.

There are tons of extensions. See the snapshot.

Screenshot 2023-09-20 at 16 01 24

link ~/.tea/stable-diffusion-webui/v*/lib/models to ~/.local/share/stable-diffusion-webui/models

can do though sd-webui puts a bunch of stuff in there itself on install so we'll be moving pkg data to userland which we try not to do. Might be a way around that…

The problem is that so much of stable-diffusion is userland changes. If you aren't familiar with https://civitai.com/ you should look at how customizable this is.

mxcl commented 1 year ago

SDXL is the model right? Yeah we should fix this to grab that by default.