search

taesungp / contrastive-unpaired-translation

Contrastive unpaired image-to-image translation, faster and lighter training than cyclegan (ECCV 2020, in PyTorch)
https://taesung.me/ContrastiveUnpairedTranslation/
Other
2.14k stars 410 forks source link

Evaluation of Cityscapes #104

Open fnzhan opened 2 years ago

fnzhan commented 2 years ago

As stated in the paper, DRN is used to evaluate the mAP, pixAcc. It seems you train a new model to conduct the evaluation and I have several questions regarding it.

  1. Why not using the pre-trained DRN model provided in the repo? such as drn-d-22-cityscapes.pth?
  2. Which set of cityscapes is used to train DRN, the training set or the testing set?
  3. Can you provide the DRN checkpoints trained by yourself for reproduction? Thank you. @taesungp
fnzhan commented 2 years ago

I evaluate the results with the pretrained DRN model, and find the mAP using drn-d-22-cityscapes.pth is only 21, but the mAP using drn-d-105-ms-cityscapes.pth can match the reported mAP in the paper. Can clarify the evaluation problem? @taesungp .