search

advimman / lama

🦙 LaMa Image Inpainting, Resolution-robust Large Mask Inpainting with Fourier Convolutions, WACV 2022
https://advimman.github.io/lama-project/
Apache License 2.0
7.99k stars 849 forks source link

can lama generate various result for one same pair of mask and image? #176

Closed universewill closed 1 year ago

universewill commented 1 year ago

can lama generate various result for one same pair of mask and image?

cohimame commented 1 year ago

Hello!

The answer is No, because lama generator has deterministic structure: it's simply a convolutional autoencoder with no noise injections.

tommycwh commented 1 year ago

I see that there is an option noise_fill_hole in the trainer. If I enable it would LAMA learns to generate image in a non-deterministic way?

https://github.com/saic-mdal/lama/blob/6bb704738d4e791106d8e87099d80831999901fc/saicinpainting/training/trainers/default.py#L61-L65

windj007 commented 1 year ago

We tried that in our early experiments and it did not lead to any significant diversity: there is a strong supervised reconstruction loss at the core of the method, which penalizes any variations. For pluralistic generation, principally different methods are needed (purely adversarial training like in CoModGAN; or autoregressive MLE models like VQGAN+transformer or diffusion)

tommycwh commented 1 year ago

By strong supervised reconstruction loss, are you referring to the L1 loss only? I see in some of your config files that the weight given to the L1 loss on missing areas is 0. I think it means that basically L1 loss is not actually effective? I am just wondering if I disable the supervised reconstruction loss(es), could I train LAMA for pluralistic generation?