exx8 / differential-diffusion

336 stars 17 forks source link

How to do outpainting correctly? #26

Open darshats opened 1 month ago

darshats commented 1 month ago

Hi, Thanks for the great work! My question is if this approach be used to do outpainting? Say I have an object without any other surrounding context, I'm having trouble getting it to fill up the context based on prompt. Below on left is the object. The diffdiff map is the second image and outputs are the right two images. The trouble is it always adds another object of the same type in the spatial location that is masked. I used a binary diffdiff map as its ok to outpaint anything as per prompt. Using a gradient mask based on distance also doesnt help as areas close to the object also need complete infilling.

image