search

DrCyanide / cyanic-sd-krita

A Stable Diffusion plugin for Krita, compatible with A1111 and SD.Next
70 stars 4 forks source link

Feature request: Lora selector for prompts #8

Closed Miraihi closed 11 months ago

Miraihi commented 1 year ago

Basically, the title. Currently I need my webui or at least the file manager to find the names for the LorAs to use in prompts. It'll be neat to have a dropdown list or something (Even better - with weight selector).

DrCyanide commented 11 months ago

Writing this down as both an update and for potential future debugging.

Extra Networks - the catch all term for Loras, Textual Inversions, Hypernetworks (and maybe others?) - has some API shortcomings. I can call different APIs to get the names of these models, and that should be enough to let users add them to their prompts easily, but that's where the good news ends. The QoL improvements surrounding these Extra Networks, things like preview images or descriptions, aren't easily accessible via the API. If I know the path to the preview image there's an API request to get the actual image, but there's no API call to get the path, and there's no API call for the description.

But that's not to say it's impossible to get the information. There's an API call - /sd_extra_networks/get-single-card - which has the description and preview image URL in it. Which is great! But... the API call doesn't return the data in JSON format, it returns it as HTML that's only intended to be used in the official Web UI.

In order to add the Preview Images and Descriptions (which I personally use to store trigger words) into Cyanic SD I need to parse the HTML. There's a Python library - BeautifulSoup - that makes this super easy, but I don't think Krita will install that for users. So I'll have to cobble together some regex or my own HTML parser to grab the data. But that doesn't avoid the second, and more dangerous, problem - the HTML could change at any time. While I'm pretty sure they will keep class="preview" for the preview image and class="description" for the text description, there's no guarantee. It reminds me of how auto-sd-paint-ext would tap into internal methods that weren't meant to be used, but admittedly it's less critical if these aspects fail. Still, I'll need to be extra careful to write the code so that 1) the extension can survive if the response changes, and 2) it is clear to the end user that the plugin code needs to be updated to get these parts working again.

Miraihi commented 11 months ago

It'll be fine to me if I can just provide the paths for extra networks at least via config file if not the proper ui. If it's going to be of any help of course.

DrCyanide commented 11 months ago

... oh, this is lovely. I'm looking at the API calls for A1111 and SD.Next here, and they diverge wildly.

SD.Next has /sdapi/v1/extra-networks which returns the name, type (lora/embedding/whatever), and preview image or link to it, as well as /sd_extra_networks/description (purpose is self evident), and /sd_extra_networks/info which includes activation text and preferred weight. In short: SD.Next actually supports accessing extra networks and building support into the plugin!

However, A1111 doesn't support any of those (I had to look again, incase I missed something), and SD.Next doesn't support the /sd_extra_networks/get-single-card call that I was thinking of using as a work around for getting descriptions.

Because the two APIs are so different here, I'll have to identify which type of server the plugin is connected to, then make separate calls to get the data, then compile that data into a common format that the plugin can use. That's... not going to be fun, to say the least.

The path of least resistance here is to just list the Lora names, but because I use the descriptions so often I hate cutting them out.

Miraihi commented 11 months ago

Having a list of names will already be of a great help.

DrCyanide commented 11 months ago

The basic lists are added. No descriptions, no preview pictures. I'm going to go ahead and close this request, because I don't think I'll be up for handling the description mess anytime soon (maybe A1111 will add an API to support it in the meantime).

Miraihi commented 11 months ago

Great, it's literally what I wanted. Thank you!