search

StoryMY / take-off-eyeglasses

Official pytorch implementation of paper "Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data" (CVPR 2022).
127 stars 17 forks source link

Questions about reproducing the training phase #5

Open PKUWison opened 2 years ago

PKUWison commented 2 years ago

This work is interesting and well-done! The inference code is clear, but I have trouble reproducing the traning phase. I wanna ask several questions.

  1. Could you release the training code?
  2. I follow all the settings avaliable in the paper to reproduce, but the inference results are excursive. Could you provide more clear settings in the implementation process.
  3. Could you provide the evaluation code for fair comparison.

Really thanks for your prompt reply. Thanks again!

PKUWison commented 2 years ago

Also, the paper doesn't mention how to split the training set and validation set. I think it is important for reproducing the training phase. Thanks for your prompt reply again!

StoryMY commented 2 years ago

Thank you for your attention! And sorry for the late reply.

  1. Training code & clear setting. The training code is a little bit dirty and is still not prepared for releasing. Instead, I add more details about training settings in the following. I hope that it could help you:
    • When training De-Shadow/Glass Network, the glass mask input is from our synthetic dataset. To enhance the ability of fault tolerance, we randomly expand or corrode the glass mask to simulate the bad prediction in real application.
    • To improve the shadow removal. The De-Glass Network is also trained to partly have the ability of de-shadow. Specifically, we feed the De-Glass Network on synthetic images with some shadow left. These images can be acquired by blending proper images in our synthetic dataset.
  2. Evaluation code. For FID, we use pytorch-fid. For face recognition, we use facenet-pytorch to get the feature vectors (image embeddings).
  3. Training & evaluation dataset. As our experiments are all cross-dataset. The split in a single dataset does not matter so much. For training data, we use all the synthetic dataset and the images with eyeglasses in CelebA, which is split by attribute label list provided by CelebA. For testing, we use part of FFHQ and MeGlass. The process of these datasets contains many manual efforts with the help of some useful tools like face-parsing to eliminate "bad" samples (e.g., sunglasses, extreme head poses). So, our split may be not the best. I recommend that you could build the testing dataset by yourself and the comparison is fair as long as all the methods are evaluated by the same dataset.
ArtyomShalagin commented 2 years ago

Hello! Thanks for your awesome research. Do you have any plans to clean up and release the training code later? This would be very helpful, I'm trying to build a smaller version of your model to make it fast enough for real-time use in my pet project but my implementation of the training phase still produces much worse results than what you have.