search

zero01101 / openOutpaint

local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
MIT License
501 stars 43 forks source link

[Feature Request] Better ControlNet Integration #200

Open seijihariki opened 1 year ago

seijihariki commented 1 year ago

Is your feature request related to a problem? Please describe.

While openOutpaint seems to work okay with the ControlNet extension, it is suboptimal to have to switch to another tab to change ControlNet parameters between generations. I don't know if we will have to modify something inside the extension to make it work with the API, but if so, we can issue some pull requests there.

Describe the solution you'd like

Preprocessor and other parameter switching for ControlNet, mainly, when the extension is installed. If we are being fancy, we can try implementing (focus on the try, as it would be difficult to do so), a 3D overlay for manual pose editing with rigs.

Proposed workflow

We would need to discuss this point.

Describe alternatives you've considered

Continue using it as-is, or sending it back and forth to the WebUI to make this type of editing.

Additional context

While it is another extension, it is quite a game changer for many. I think it would be useful to integrate, and if it is integrated in the future by AUTOMATIC1111, we can just change some API calls, probably.

This will be a medium-long term feature to implement. Created a branch controlnet to play with this for now.

zero01101 commented 1 year ago

i haven't even gotten an opportunity to try controlnet yet but holy crap does it appear to be terrifyingly powerful; in a perfect world i'd of course prefer to have first-class support for controlnet directly within webUI like instruct-pix2pix received, but yeah, if it stays an extension then so be it - i believe there's at least 2 existing extensions for it and i personally lean more towards this implementation just since it's a single-purpose extension dedicated to it

also thanks again for your recent commits of course <3 sorry i'm kind of not doing anything too useful lately lol

lmao god i just noticed you linked the exact same extension i did i am really observant

seijihariki commented 1 year ago

Yeah, that is the one I'm personally using because of its genius implementation that merges the controlnet on the fly with whichever model you are currently using, making literally every single 1.5 based model compatible with controlnet.

Also, don't worry! I have been a bit absent as well, and I have seen a bit of what you are going through.

zero01101 commented 1 year ago

looks like API support is coming for control net one way or another :D
https://github.com/Mikubill/sd-webui-controlnet/issues/63 https://github.com/Mikubill/sd-webui-controlnet/pull/194

seijihariki commented 1 year ago

That was quick! Also, we can add something like this to openOutpaint quite easily I think:

https://github.com/fkunn1326/openpose-editor

This was what I was thinking about implementing, rather than a full 3d rig editor.