search

WeilunWang / NEGCUT

Official Implementation of Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation (ICCV 2021)
46 stars 7 forks source link

Cityscapes Evaluation #1

Open veroveroxie opened 2 years ago

veroveroxie commented 2 years ago

Hi, thanks for your work.

Could you please release the drn model and your pre-trained models?

About the cityscapes, do you use the validation set given by CUT or the test set given by running prepare_dataset.py? Or do you change the training set of this dataset? (I have tested the pre-trained model of CUT, the results are very weird. When I use the validation set, the FID matches the results in their paper. But if I use the test set, the FID is around 100.)

For the experiments, do you choose the best epoch or the final epoch?

Sorry for so many questions. Best

WeilunWang commented 2 years ago

Hi, thanks for your interest.

  1. Regarding the pre-trained model, since I am on holiday, it's not very convenient for me to get them right now. I will release them later.
  2. Same as CUT, I use the validation set given by CUT to evaluate our model. This is because the semantic maps of test set is not avaliable. (They are all blank in the cityscapes dataset.) This may be why you got a large FID when you test on the test set.
  3. On the cityscapes, we choose the final epoch, which is also the best epoch.
veroveroxie commented 2 years ago

Thanks for your quick reply!

For the cityscapes dataset, CUT used this resized and JPEG-version. Do you use this tar file in your evaluation?

""" Note: the Cityscapes pretrained model was trained and evaluated on a resized and JPEG-compressed version of the original Cityscapes dataset. To perform evaluation, please download this validation set and perform evaluation. """

WeilunWang commented 2 years ago

Hi,

You can send a email to me, I will reply you with my test demo.

Best, Weilun

Lucky0775 commented 2 years ago

Thanks for your quick reply!

For the cityscapes dataset, CUT used this resized and JPEG-version. Do you use this tar file in your evaluation?

""" Note: the Cityscapes pretrained model was trained and evaluated on a resized and JPEG-compressed version of the original Cityscapes dataset. To perform evaluation, please download this validation set and perform evaluation. """

Hi,have you solved this problem ? I also have the same confusion.