Not an issue, External Prompt Editor Idea

[Hellrunner2k]

Hi.. me again ^^ .. i had an idea for an external prompt editor. there is no discussion board.. sry

is there a way to get and set the prompt widgets in krita? godot supports GET and POST natively. idk how it works.. is it like a server api with a port or do you communicate differently with krita?

[DrCyanide]

Stable Diffusion WebUI has an API that uses rest commands (HTTP GET and POST). Basically any language can communicate with that to interact with the server. None of these API calls are for getting the last prompt used.

Krita plugins are written in Python , and use a Python library to talk to the application. So it's not possible to control Krita from an outside plugin (to my knowledge). That said, you can make basic web servers in Python, and I'm aware of other plugins that establish ports to listen and post to - such as a plugin that lets Krita interact with Blender.

I can understand wanting better prompting tools (spell check, prompt history, etc), but I don't quite see the point of an external prompt editor. What value do you think that would add? As always, you're free to develop features you want to see added and submit a pull request.

[Hellrunner2k]

i could make an interactive system (Godot excels at UI design), where you could create and manage prompt snippets and combine them with a couple of clicks to the output prompt.

for instance you could describe a specific centerpiece ... an outfit... a scenario, quality description.. all in prompt snippets these would be entities with positive and neg... all instantly stored on edit in a .csv (multiple sequential autosaves incase something effes up ^^)

and you can combine them to an overall main prompt or structure and redistribute it for regional prompting or attention masks. that way you have a platform agnostic solution for your prompteneering ^^

(side thought ... masks need alpha.. i paint and blend them smoothly just for it to binary them ^^.. ANYWAYS)

does krita UI support dynamic blocks in some form? maybe it could be done in a second docker... the main UI is quite crowded as is. dockers can talk to each other right? ... color picker

i'm a total python noob tho .. i come from unreal and know Godot script... this platform agnostic thing made with godot would be nice.. cuz you could just switch it to forge or comfy too

there is a lot more in the idea sector... maybe hit me up on discord? ^^ discord ID -> .hellrunner

[DrCyanide]

You can un-dock, move, and resize the plugin (or any Krita dock) to fit your liking. Krita manages all of that, all I did was specify that the plugin should dock on the left side the first time it's installed. Krita will remember where you put it the next time it's opened.

The only way your prompt idea would be platform agnostic is to build extensions for each platform to connect to it - or if the user just copy-pastes the prompt into the platform after creating it in another program. I think what you're describing is similar to what the wildcard extension adds already, but regardless it's not something I'm going to spend my time developing.

Alpha masks is interesting, but I would have to do more research on it. I don't know if Stable Diffusion supports alpha masks or if settings like "mask blur" add alpha to the final image instead of the mask. The masks don't preserve color, so I'm not sure why you're blending the mask instead of waiting for the generated image (but admittedly that might be because I use neon green to mask so it's clear what's selected).

Right now my focus is on other projects and workflows rather than adding features to this plugin.

[Hellrunner2k]

i think i didn't get the idea across ^^ ... anyways.. the color of the mask does not matter.. the alpha of the layer does ... in comfy you can have 8bit gradient masks to influence the conditioning at gentime or as inpainting mask... like ... it's a black and white image mask type (green in comfy) as opposed to the RGB image input...

an option to use the layer alpha 1:1 and override binary conversion and feather?

[DrCyanide]

I'm going to document what needs to happen here while I've got the code up. This is more technical notes for myself (or anyone else who wants to work on this) than anything else.

krita_controller.py has the get_mask_and_image(self, mode='canvas') function, which handles converting the mask from whatever colors to black and white. It's using QImage's createAlphaMask() to create the mask, but that function is limited to 1 bit per pixel, either white or black.

It might be possible to create a grayscale mask by using pixel manipulation to change the value of each pixel, but it'll take more looking into.

[DrCyanide]

OK, so I have this implemented, but I don't think it's working how you think it should because your example was in ComfyUI, not A1111. Regardless, the code is pushed, so you can pull it and give it a shot.

So this is the example mask I tested with. I was able to generate it, using the method described above, copies the alpha values as grays, looks great!

However, the results from such a mask look more like a hard edge in the results, not a gradient. Original Background: Inpainted Results:

It might be hard to tell side-by-side, but if you toggle between them then you can see that it looks closer to a hard line between where it changed it and where it didn't.

Here's a reference with the overlaid mask. You can see it is painting in areas that have a good amount of transparency, but it's not clear where the cutoff is. I suspect this will cause more issues with things not being inpainted where you want them than it will help them.

But a few days ago I implemented A1111's "Soft Inpainting" into the Inpaint tab. That will probably give results closer to what you're looking for. I don't know how to get the best results with it, so check out documentation or YouTube videos on it.