search

Bing-su / adetailer

Auto detecting, masking and inpainting with detection model.
GNU Affero General Public License v3.0
4.16k stars 320 forks source link

[Feature Request]: Ip-Adapter controlnet support #730

Open merket opened 2 weeks ago

merket commented 2 weeks ago

Is your feature request related to a problem? Please describe.

When trying to create a character grid, using ip adapters for influencing the faces with a formerly created character, adetailer totally bypasses that and makes it impossible to use ip adapters for a prompt.

Describe the solution you'd like

ADetailer doesn't seem to support IP-Adapter controlnets, but hopefully it will in the future. It does allow changing checkpoints and even have a controlnet model input but it doesn't allow ip-adapters. Please make it so that we can supply source images for adetailer to build on.

Describe alternatives you've considered

No response

Additional context

No response

Bing-su commented 2 weeks ago

Well, I'm still not sure about the usefulness of other controlnet models for inpainting.

At least the ip adapter isn't the type that uses the generated image as a control image, is it?

lhucklen commented 1 week ago

looks like he did something in dev branch and referenced this, I think our ideas and desires align. Hopefully, @Bing-su has some ideas.

701

lhucklen commented 1 week ago

Well, I'm still not sure about the usefulness of other controlnet models for inpainting.

At least the ip adapter isn't the type that uses the generated image as a control image, is it?

I have played with it via pass through and it does have the desired effect at low denoise value.

I mentioned some ideas in #701, looks like you did something in the dev section, would love to hear an elaboration on how you used that idea. have you had any chance to consider this for full main branch?

ceacc711e459f685d8e0795bc10308df1baf505d

merket commented 1 week ago

Well, I'm still not sure about the usefulness of other controlnet models for inpainting.

At least the ip adapter isn't the type that uses the generated image as a control image, is it?

As I said, the Ip adapter could help the character consistency. For example, I am using ip-adapter ip-adapter_xl [4209e9f7] and / or ip-adapter-plus-face_sdxl_vit-h [c60d7d48] to influence of my outcome and before (or without using) the adetailer I can achieve the result I desire, meaning a face similar to my source image. But since adetailer doesn't utilize the info or have it's own ip-adapter, it totally bypasses that process and creates a totally different outcome. Of course I can lower the denoising strength to stay closer to my prior results, but then adetailer does not really does it's job of fixing or detailing the face.

I hope I explained myself clearly. I have seen other people mentioning and requesting the ip-adapter for support for adetailer on youtube as well. We just think it would be a useful addition.