light-and-ray / sd-webui-replacer

A tab for sd-webui for replacing objects in pictures or videos using detection prompt
193 stars 11 forks source link

Support for the Inpaint difference extension is very unstable #32

Closed PladsElsker closed 6 months ago

PladsElsker commented 6 months ago

Hello, I just saw that you added support for the Inpaint difference extension!

One problem I see is that you are using the compute_mask function directly. I just updated it like a week ago to add a few parameters to it, so in a sense, it is not a stable way to do this.

If I modify the arguments passed to the function (I still have a few things I want to do, so very possible), this is gonna break on your side, and you don't want to spend your time fixing these sort of issues.

The better way to implement this would be to implement it in a more implicit way. Probably not possible, but calling compute_mask with a variable amount of parameters (*args) would be better, for example. You could also highjack it, and run your extension when the compute_mask function returns? I really don't know to be honest, but something like that.

The same problem happens for using the gradio elements of the Inpaint difference extension directly in yours. Basically, you want to stay as agnostic as possible to how other extensions work.

Inpaint difference relies on a library I'm maintaining, sdwi2iextender. The point is to unify how img2img tabs are added. Maybe your extension would benefit from using it? Instead of being a full fledged tab, it could maybe fit into an img2img tab? I don't know, it sounds like it could when I look at the readme.

light-and-ray commented 6 months ago

Thanks for your message

Instead of being a full fledged tab

I like to use a dedicated tab, I think regular img2img tab is too complicated for my extension. The benefit of using all img2img scripts was questionable because it seems no one fit for inpainting except controlnet and tiled vae. And placing replacer tab next to other img2img tabs would break the idea of maximum fast inpaint. Later I decided to add custom mask and inpaint diff into advanced options just to have they there and take advantages of hires fix and cn inpainting

I just updated it like a week ago to add a few parameters to it, so in a sense, it is not a stable way to do this.

Yes, I understand it. It's an inevitable problem with using of any external extension. I think the best way I will fix it if you change something, like I'm doing for controlnet. Do not care about it. At least your development won't be endless 😃 idk how much new features are possible to add in your original idea

with a variable amount of parameters (*args) would be better, for example

It's impossible because if takes parametres from gradio's input

You could also highjack it, and run your extension when the compute_mask function returns?

This can solve a problem when an author changes returned variables, but for other cases it won't fix it as I see

PladsElsker commented 6 months ago

I see, if you don't mind having to maintain the arguments, there's no problem then. I'll close the issue 👍 For new features in Inpaint difference, you never know! I keep thinking of new things to add to it :p
For example, I was thinking of adding an option to only return the inpainted masked region, and leave the rest of the image with full alpha when returning the image (to use as a layer in clip studio paint).