search

kozistr / improved-edge-connect

EdgeConnect: Structure Guided Image Inpainting using Edge Prediction, ICCV 2019 w/ some my ways :)
https://arxiv.org/abs/1901.00212
Other
14 stars 6 forks source link

How is this improved? #1

Open Sicily-F opened 3 years ago

Sicily-F commented 3 years ago

Hi there,

I am very interested in using this repo, and I was wondering if you could provide an explanation of how it is the new improved version? I've been struggling to get Edge-Connect to work on my own data for a while.

Any tips would be great!

kozistr commented 3 years ago

First of all, thanks for your interest : )

Actually, It's been a long time while I work on it, so I can't remember everything, and maybe the whole works weren't uploaded to this repo because it's related to the company work.

Anyway, what I tried were

  1. perceptual loss (style loss part in this repo) : code
    • Based on ESRGAN, when calculating perceptual loss, It's better to use the output of the convolution layer instead of the output of the activation layer (you can find the details in the paper). So, I tested it on my data and it works better!
  2. network modules : code
    • I adopted Instance Layer Normalization (a.k.a ILN), and several techniques I guess.
  3. custom mask
    • I didn't upload the code, which makes a mask, I made a custom mask logic, which fits in my task, and it gives a good result.

That's all I have tried! You can apply other techniques and SOTA image-inpainting architecture (2021). If you have any questions, feel free to leave a comment : )

Thanks in advance! Kim

Sicily-F commented 3 years ago

Hi Kim,

Thank you so much for your response - this is great and so quick!!!

I will definitely try those things for sure. Can I ask if you could share your config.yml file for training on your own data? I would like to use the pretrained model, but train it further on my data, in an application of transfer learning. This is the tricky part for me!

Many thanks, Sicily

kozistr commented 3 years ago

Unfortunately, I don't have any settings & models now because it related to a company project a few years ago : ( Maybe you can train your model on the big dataset (like CelebA, Place365, etc) first, then fine-tune on your dataset!