DrCyanide / cyanic-sd-krita

A Stable Diffusion plugin for Krita, compatible with A1111 and SD.Next
68 stars 4 forks source link

Inpaint area: Only masked doesn't work correctly #1

Closed Miraihi closed 10 months ago

Miraihi commented 11 months ago

First of all, thanks for your efforts, @DrCyanide. I tried your extension and it works in most cases better than the auto-sd-krita. I like that I can use my saved styles, and of course the proper access to ControlNet with Adetailer is the gamechanger.

Unfortunately, for my main use cases, while the "Old way" of using the inpaint function works (I mean "Choose the selection area with the desirable resolution, mask it, then generate"), the "Inpaint area: Only masked" always uses the resoluton of the whole image as the base, what makes things inflexible. Also the new generated layer is never placed at the correct position, always off.

I think we need better resolution control, as well as the option to use upscalers of choice for img2img generation.

EDIT: Nevermind, the expension has been misbehaving due to my inexperience. Only Masked actually works properly.

Still the resolution control and more hassle-free mask control (So I don't have to renew the mask each time I make a change manually) would be nice to have. Also Krita crashes (Without error report) in most cases when I try to interrupt the generation.

DrCyanide commented 11 months ago

There was a bug with the position of Inpainted areas if you clicked the "Use Layer" option, where it would put everything in the upper left corner. This would've effected Img2Img if anyone had a smaller-than-canvas sized layer. I've just pushed an update that should fix that now.

I like the idea of an "Always update mask from this layer" option, but figuring out how to convey that in the UI without extra clutter is tricky. Maybe I can replace the "Use Selection" button with a "Refresh Mask" checkbox? I don't think a selection is as useful in Inpaint mode, because it's unlikely you'll want just part of a mask you drew to be used.

I'm not seeing an Upscaler option in the web-ui for Img2Img, just a resize option (which I believe uses the latent space). I think it makes the most sense to put upscalers (and face restore, and background removal) in a new Extras page, rather than try to crowd the existing Img2Img or Txt2Img with them.


Resolution control is trickier than I'd like, so I'd appreciate some input on it.

My preferred option would be that if someone specified they wanted an image bigger than the canvas (either from upscaling or Hires Fix) that image have a Transform Mask would be added to it so it could be scaled down non-destructively to fit the canvas. Due to the limitations of the Krita 5.1.X API I can create the Transform Mask, but I can't scale it. So what should happen instead? Should the canvas be automatically resized to fit the bigger layer, or should the layer just extend off the canvas? How would that interact with existing size selections?

Speaking of selections - I haven't documented this, but if you have a selection box made while you do Txt2Img it'll use that selection size instead of the canvas size. I could have the plugin make a selection box that's a requested size (so long as it's smaller than the canvas).

I've also considered allowing users to make rules about the size in the settings tab. Stuff like "Don't generate images smaller than 512x512. Generate them at that size, then scale them down to fit" or "If generating an image larger than 768x768, automatically apply hires fix/upscaler X". I'm torn between how handy it would be to not have to worry about resolution at all, vs wanting full control over everything. Fooocus vs A1111 design principles.

Miraihi commented 11 months ago

So what should happen instead? Should the canvas be automatically resized to fit the bigger layer, or should the layer just extend off the canvas? How would that interact with existing size selections?

Well, "Layer extending off canvas" certainly doesn't sound right if we're talking about inpainting. A great resizing system has been an awesome part of the old extension, but not sure if something can be reused.

I want to stress that your extension is already superior to the sd-auto-krita.

DrCyanide commented 10 months ago

Upscaler support has now been added.

I'm going to go ahead and mark this issue (which was originally about masks not working) as complete. If you want to add the other ideas in here as separate feature requests that'd be a better way to manage them.