search

zero01101 / openOutpaint

local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
MIT License
499 stars 43 forks source link

Outpainting using saved image #204

Open Ralph224 opened 1 year ago

Ralph224 commented 1 year ago

What happened?

Hi,

I'm loading an image from "PNG Info" to OpenOutpaint. In OO, I choose model, prompts, SD settings and so on. Click on a canvas to get everything going. Once I get the results, the original picture stays there but the new added outpainting looks like it never took the original picture into consideration. It brings up nice pictures but doesn't flow with the existing picture.

Image I'm using are with people and changing background is the main goal.

Is it something I'm doing wrong or it's an actual bug?

Steps to reproduce the problem

Load existing PNG from PNG Info Send to OpenOutpaint In OpenOutpaint, run with prompt

What should have happened?

The outpainting should have compare the existing image to be able to extend the missing piece.

Commit where the problem happens

NA

What platforms do you use to access openOutpaint?

Windows

What browsers do you use to access the UI ?

Microsoft Edge

Browser Extensions/Addons

NA

AUTOMATIC1111 webUI Commandline Arguments

NA

Additional information

NA

joshyy1312 commented 1 year ago

Edit: I checked right now and it doesn't work anymore, even with inpainting specific models. It's as Ralph224 said, the model generates new images without trying to "continue" the current given image.

Old post: From my experience, this happens when using a normal model instead of an inpainting model.

Normal models can be used with img2img to modify the existing image but you have to use a dedicated inpainting model to extend the original image in the way you describe. And then use your normal model to img2img the extended zone in the correct style.

While in OpenOutpaint, In stable diffusion setting, the correct models to extend your image appear in green in the list. (I usually use 512-inpainting-ema)

seijihariki commented 1 year ago

Would it be possible to record a simple video for this? I cannot really replicate the issue in question... If you can also enable debugging and send the downloaded images here...

joshyy1312 commented 1 year ago

I tried to reproduce it now and I can't.

The only difference I see with before is that I closed the browser (firefox).

As I got the habit of refining big images (4k/8k) with the extension's img2img, would it be possible it's something that happens when working for a long time? Maybe memory leak/fragmentation?

I'll enable debugging next time I do a long session with big pictures (tonight probably).

seijihariki commented 1 year ago

Thank you. Remember to save your workspace not to lose any progress to this issue.

joshyy1312 commented 1 year ago

This morning it happened again, so I enabled debug and recorded the process. The video and both debug images are in the zip.

It wasn't corrected when I clicked "reset to default" in debug but it WAS corrected when I clicked the "refresh openoutpaint" blue button at the top of the canvas.

Thank you for looking :) outpaint bug 204.zip

PS: Please don't mind the alien parasitic worm, it's for the rpg I DM :p