zero01101 / openOutpaint

local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
MIT License
499 stars 43 forks source link

[Feature Request]: Mask based on "rembg" result #217

Closed FatGuy84 closed 1 year ago

FatGuy84 commented 1 year ago

Is your feature request related to a problem? Please describe.

Reworking a background around a subject with the mask brush is always very fiddly, inaccurate and time consuming.

Describe the solution you'd like

A proper possible solution high accuracy would be to use the "Remove Background" function of extras on a selected area and invert the result as a mask.

Proposed workflow

Select Image solution

  1. Select "Select Image"
  2. Check "mask without background"
  3. Select background removal model (or just u2net)
  4. Click "create mask"

Describe alternatives you've considered

Import Mask solution This solution would require to select an area, save/send it to extras, remove background and send it to openOutpaint

  1. Select "Stamp Image"
  2. Check "mask image"
  3. Stamp mask

Additional context

Thank you for taking the time to read and consider my feature request

I recently stumbled upon your project, and I must say that I am absolutely fascinated by the incredible work you have done, I am genuinely excited to see how it evolves and grows in the future.

zero01101 commented 1 year ago

so it'd be awesome to implement and would let me get rid of my extra-janky background carving option i slapped in openOutpaint a few days back, but unfortunately there's no API endpoint to control it :/ if it gets API support i'll gladly revisit the idea :)

FatGuy84 commented 1 year ago

@zero01101 Thanks for following up! Pity, that no API exists.

Any chance to at least support a mask stamp?

Would help a lot already to

  1. Send a selection to Extras
  2. Remove BG
  3. Send the subject back as mask
  4. Stamp the mask like an image

What do you think?

zero01101 commented 1 year ago

hey, sorry for missing this forever ago, but it's actually kinda sounding like segment anything's going to be getting some API support which will probably take care of things quite handily :)

FatGuy84 commented 1 year ago

Hey, that sounds interesting.

FatGuy84 commented 1 year ago

@zero01101 Do you mind to reopen this request?