search

hafriedlander / stable-diffusion-grpcserver

An implementation of a server for the Stability AI Stable Diffusion API
Apache License 2.0
172 stars 23 forks source link

getting inpainting and outpainting to work #3

Closed imperator-maximus closed 2 years ago

imperator-maximus commented 2 years ago

Hi, I am using same settings as for Stability AI grpc but Inpaiting and Outpainting (=both mode inpaint on server console) does not work at all. It is obvious that this server uses inverted mask in comparison to Stability AI. So I provided inverted version but it still does not work. I played around with different strength parameters. For transparent parts I provided grey fill because it is required in Stability AI. I tried it with transparent but it also does not work. image I have no problem to change anything but it would be nice to get an idea about how to do this? Another question in this context: an option to generate alpha maskl from transparent parts on server would be also nice. Thank you!

hafriedlander commented 2 years ago

I haven't tested the mask direction or strength versus grpc.stability.ai yet (unfortunately it's not documented in the proto).

The expectation for outpainting at the moment is: white for outside the image, black for inside the image, grey only for blending (and strictly inside the image - anything with no image data underneath must be pure black). I'll have to reverse-engineer what the official API uses.

imperator-maximus commented 2 years ago

I tried white and black:

unknown_w unknown

Results are even more worse now. But this is init img btw. For mask I already had black white.

imperator-maximus commented 2 years ago

here you get all informationen how Dreamstudio handles it: https://github.com/Stability-AI/stability-sdk/issues/62#issuecomment-1259834416 it is basically only if (png) scheduleParameters.setStart(mask ? 1 : 1 - this.$refs.editor.generationScheduleStart / 100); for the strength plus images. And with await editor.renderSVGToPNGDownload("image"); await editor.renderSVGToPNGDownload("mask"); you can save these images.

imperator-maximus commented 2 years ago

I am getting great results now with sampler=0 (should be DDIM )

imperator-maximus commented 2 years ago

I am closing this now because it seems to work and might open new tickets on detail problems. Thank you! :)