butaixianran / Stable-Diffusion-Webui-Civitai-Helper

Stable Diffusion Webui Extension for Civitai, to manage your model much more easily.
2.4k stars 287 forks source link

Feature Request: Store the descriptions of the model inside the .info file #65

Closed SeverianVoid closed 1 year ago

SeverianVoid commented 1 year ago

Would be nice if you could include in the .info file and then displayed somehow on the extra networks page, maybe a button you hover over or something the written descriptions that authors of models and files on civitai provide, since quite frequently they include a lot of details about how the models work or extra steps you might need which aren't then available anywhere else.

The button to open the page the model was from is convenient to get there, but sometimes I have found that to not work and it doesnt actually open any page to the model and I am almost always opening it just to check the description

butaixianran commented 1 year ago

saved description won't be updated, just click url button to open civitai model's webpage.

If it can not open civitai url, means you don't have a model info for this model. Check SHA265 section of github document for the reason and solution of this.

SeverianVoid commented 1 year ago

And what happens if the model is removed from the site? Or the site itself closes down? That description would then no longer be available on the page since the model page would no longer exits. This is probably the most important reason for why storing the model description in the .info file would be beneficial.

The number of model uploaders who use the description for extra information is just too much to not include it in the model info file most importantly for the event that the model page is no longer available.

SeverianVoid commented 1 year ago

As for the model link not working, when I click the web link button in the extra networks model page the error output I get in the stable diffusion console window looks like this.

image

Civitai Helper: Load model info of d3c225cbc2e0ed3ac8032e05e6fabe5acfe81e2e3d18e23389e84b9cb5ce127f in ckp Civitai Helper: Can not find model info file: F:\Neural_Networks\Automatic1111_Git\stable-diffusion-webui\models\Stable-diffusion\ d3c225cbc2e0ed3ac8032e05e6fabe5acfe81e2e3d18e23389e84b9cb5ce127f.civitai.info Civitai Helper: Failed to get model info for ckp d3c225cbc2e0ed3ac8032e05e6fabe5acfe81e2e3d18e23389e84b9cb5ce127f

For whatever reason its trying to use the model hash as the name of the .info file rather than the matching model name for the .info file. The info file exists right next to the model and has all the correct information in it, but the web link button doesn't work since its looks like its trying to call for the info file using the hash instead of the model name.

image

The sha256 hash in the model info file matches what its trying to look for in the console, and has the correct web links and information.

I think I may also only be having this with full models, the web link button for LORA files I don't think I have run into the same problem

SeverianVoid commented 1 year ago

Just did a bit more testing, and with LORA, TI and Hypernetworks I had no issues loading the model page using the button and the output in the console looks like I would expect calling for the model name

Here is the output hitting the web url button for a LORA

image

So it only seems to be affecting the checkpoints section of the extra networks page, and it seems to happen with every single checkpoint I have tried, as they all show up in the console window looking for the hash value

butaixianran commented 1 year ago

I downloaded the "/comicDiffusion_v2.ckpt" model you mentioned here, and can open its url corrrectly without any issue.

The reason your extension can not find this model info, is somehow, your SDwebui, can not get this model's file name.

For this model, SDwebui should have a string like this saved in its model card:
/comicDiffusion_v2.ckpt d3c225cbc2e0ed3ac8032e05e6fabe5acfe81e2e3d18e23389e84b9cb5ce127f
As you can see, it's this mode's file name + hash code.

Then this extension, will get the file name part and use that to get model info file.

But somehow, your error msg tells me, your SDwebui can not read this model's file name. In your SDwebui, this hidden string is saved as:
d3c225cbc2e0ed3ac8032e05e6fabe5acfe81e2e3d18e23389e84b9cb5ce127f
As you can see, there is no file name in it. That's why this extension can not find its model info file.

So, you need to find out why your SDwebui can not read this model's file name.

I'm guessing, you put this model in a folder, your SDwebui's extra network can not read. So, it can not find this model, but it remember its hash code. For example, a model saved on a mobile drive, and it is offline when SDwebui checking this file.

SeverianVoid commented 1 year ago

All the model files are located: stable-diffusion-webui\models\stable-diffusion All the LoRas are in: stable-diffusion-webui\models\LoRA All the hypernets are in: stable-diffusion-webui\models\hypernetworks and the TI are in: stable-diffusion-webui\embeddings

The stable-diffusion-webui folder is on a local SSD internal hard drive. So the full path to the checkpoint folder looks like this F:\Neural_Networks\Automatic1111_Git\stable-diffusion-webui\models\stable-diffusion image

All the model checkpoints show up in the extra networks tab, they scan using the extension just fine and every single one has a .info file and preview file. I even deleted every single .info file and hit the scan button on the extension tab and it recreated them all perfectly fine you can see the dates for those are all moments ago. Clicking on the model in the extra networks tab also correctly causes the model to load.

image

Is there a file somewhere that stores the SDwebui extra network data that might have gotten screwed up somehow that I can delete and have it recreate? you mentioned that it saves that info in the model card somewhere, how would I clear that and have it rebuild?

I checked and both the extension and the webui are fully updated with git pulls

butaixianran commented 1 year ago

hi, there is nothing to do with this extension about this issue.

You see, the string is saved by SDwebui, this string's name is called "search_term". It is created by SD webui, and saved into every model card by SDwebui's extra network.

This extension can only read it. So, no matter how you delete or re-scan models, won't change anything. Your SDwebui is the issue here.

You need to figure out, why your SD webui can not create it right. One thing you can try is, Every time you change something then click "Refresh" button, it will re-create this string again. So, you can remove or change any checkpoint model's preview image, then click "Refresh". After that, try to open its url, see if it works again.

Also, you need to find out if only this model has this issue, or, all checkpoint models has this issue. Lora won't have this issue, only checkpoint models.

One thing I feel could be an issue is, you have so many extensions here. other extensions may break it. You may need to check it one by one.

SeverianVoid commented 1 year ago

I tried duplicating one of the model checkpoints and changed its name, re-scanned with the extension and it loaded up correct previews and .info file from the website identical to the model I copied, refreshed the extra networks list, and I get the same results still the copied model also brings up the hash being used instead of the model name. image

Its happening with every single checkpoint model in the extra network panel not just the one model

I will try disabling extensions to see if that is what might be causing issues.

butaixianran commented 1 year ago

OK, here is a way you can directly see this "search_term" in your web page.

Right click a checkpoint model card, select "Inspect Element" from the menu. It will open it's html code page.

Spread the "div" card node, until you find "search_term" node. You'll find the string there.

You need to check if file name is in it. It should be file name + hash code. Like following.

search_term

If file name is in it, you need to paste your search_term here, we'll see why your search_term doesn't work.

Also, have you tried re-launch SD webui ? Not just Reload UI.

SeverianVoid commented 1 year ago

I have disabled every single extension apart from the civit helper extension and then restarted the entire thing not just reloading but closing the command window and relaunching the webui-user.bat, issue remains.

I pulled up the html like you showed and it doesn't seem to have the same format of information in the search term just the hash value with no quotations or file name. It seems like the other url lines in the class style are also slightly different?

image

Tested in chrome as well in case it was something caused by Firefox image Same issue

butaixianran commented 1 year ago

You did very well, you proved there is no file name in your model's search_term.

That mean's 2 things:

Here is something gonna help you:

Go to bottom of SDwebui's page, there is a commit number. that string, is a url. Copy the whole url to here. We'll see which version of SD webui you are really using.

Or you can submit a bug report to SD webui's issue list, with above sceenshot, ask them why there is no file name in your search_term

SeverianVoid commented 1 year ago

When I launch the webui

Commit hash: a9fed7c364061ae6efb37f797b6b522cb3cf7aa2 image

From the bottom of the page image

Is it possible that its some setting in the webui settings thats causing it?

butaixianran commented 1 year ago

ok, good news is you are using latest version of SD webui.

Bad news is, that means this is a bug of SD webui.

In SD webui, it uses following code, to get the file path and put it into search_term:

    def search_terms_from_path(self, filename, possible_directories=None):
        abspath = os.path.abspath(filename)

        for parentdir in (possible_directories if possible_directories is not None else self.allowed_directories_for_previews()):
            parentdir = os.path.abspath(parentdir)
            if abspath.startswith(parentdir):
                return abspath[len(parentdir):].replace('\\', '/')

        return ""

In above code, following line, is the reason, your search_term doesn't have a file name: if abspath.startswith(parentdir):

In this line, abspath is your model's full path. parentdir is your model folder. But in your SDwebui, it thinks your model is not in a model folder SDwebui allows.

So, it won't add the file name into search_term.

Yes, settings and command line arguements can cause some trouble. If you are using some comannd like "--data-dir", you better remove them.

Still, this should never happen, so, it is a bug of SD webui.

butaixianran commented 1 year ago

There is a way to help you find out which folder name caused this, if SDwebui's team won't help you.

Open your SDwebui/modules/ui_extra_networks.py as txt. Go to line 59.

Change line 59 and 60, from this:

            parentdir = os.path.abspath(parentdir)
            if abspath.startswith(parentdir):

To this:

            parentdir = os.path.abspath(parentdir)
            print("model abspath: " + abspath)
            print("model parentdir: " + parentdir)
            if abspath.startswith(parentdir):

Then re-launch your sdwebui, go to extra network's checkpoint model page, and view console log's output, to find out which folders are allowed, and why your model is not in one of those folders.

SeverianVoid commented 1 year ago

Changed those lines of code to what you had, relaunching in the console this is what it shows now.

image

So I have no idea whats going on.

I just started a full fresh install of the webui to a new location on the same drive in the hopes that if it is a configuration issue or some setting that is breaking things that it wont show up in a fresh install

butaixianran commented 1 year ago

Your screenshot already shows the reason, one path is "stable", the other is "Stable", "s" is different in these 2 paths.

Snipaste_2023-03-22_08-49-01

There is no reason to re-install, just change the "s" to uppercase.

And why your checkpoint folder's name is "stable-diffusion"? Why the "s" is lowercase? Have you re-named it before?

SeverianVoid commented 1 year ago

So I just checked, and in the install the folder it is indeed lower case... never touched the name of it before though but I have also been using this same install for a very long time.

My first images in the output are from 6 months ago image

But now that I think about it at some point I had to move all the model files since they used to be stored in a different place months ago, so maybe when creating the new folder structure for the current model locations it was done with a lowercase s, hypernetworks is also lower case but those didn't have the same issue.

image

So weird especially that it worked most of the time for almost everything and the case only threw things off for that one aspect, switching it to a S has indeed solved the issue, thank you so much

SeverianVoid commented 1 year ago

Greatly appreciate your help in figuring out what was going on, never would have figured out that letter case of the model folder is what was causing the problem.

I do still think that storing the model description as a part of the .info file would be a good thing to have in the event the model is removed from civitai, or if the civitai site is down.

Even if there is no way to display it in the webui, just having it in the .info for record keeping would be enough.

butaixianran commented 1 year ago

Model folder's name must be exactly the same as official github project.

Search term is used by extra network, so, there would be something failed when using extra network, you just didn't notice that. When it failed, it didn't tell you.

And download description's feature will not be added. Just open civitai's url.

This request is closed.