search

advimman / lama

🦙 LaMa Image Inpainting, Resolution-robust Large Mask Inpainting with Fourier Convolutions, WACV 2022
https://advimman.github.io/lama-project/
Apache License 2.0
7.61k stars 820 forks source link

Image2Image model #93

Closed armheb closed 2 years ago

armheb commented 2 years ago

Congrats on your great work and thanks for sharing your code. I'm using this model for an image2image translation. but after training for some time I'm losing the details from the input image in the predicted results. do you have any suggestions on how to modify the loss functions for better detail perservation? Thanks.

windj007 commented 2 years ago

Hi! Sorry for the late reply.

do you have any suggestions on how to modify the loss functions for better detail perservation?

The answer would depend very much on the task and the dataset. Given more details, I can probably recommend something.

armheb commented 2 years ago

Thanks for your reply. Basically, I'm using the lama to get rid of some artifacts in my images. Below is an example image batch0000107 . from left to right is the input image, target, and lama output. as seen above I'm losing some of the details in the texture and images on the shirt. I was hoping to modify the loss functions to better preserve these features in the input image. Thanks.

windj007 commented 2 years ago

I see.. I think that this task is pretty different from what Lama was deisgned for - so you probably need to change not only losses, but also architecture and data generation process. There are plenty of possible options to try from stochasticity in generator to custom perceptual losses.

Also, it seems that FFC is a bit meaningless in that task - in inpainting it increases receptive field and helps to capture patterns - but in this example mostly local information is relevant.

armheb commented 2 years ago

Thanks, Yes I have changed the data generation process and architecture. I'll also try custom perceptual losses too. Thank you.