Reproduce segmentation results

[xiaoqianjia]

Hi When I try to reproduce the cycada to cyclegta5 to city images, it will stop running early,
shows "No suitable discriminator found -- returning" but at this time mean IOU is about 26%.

Does anyone meet the same problem, or Is it normal for cycada? Do I make some mistakes?

And for train_fcn_adda.sh, the paper said lambda will influence the mIOU results, it means the lambda in cyclegan part? Will the lambda_g & lambda_d in feature adaption influence the mIOU results? Thanks

[Luodian]

@jhoffman I encountered the same problem. the domacc value is set 60. But until the end of training, the discriminator network's accuracy can't reach that high.

[Luodian]

I tried to set the value at 55, it works better but I am not sure whether it's best. Another question, during training, the mIOU keeps degrading from 90 to a stable value. Why would this happen?

[jhoffman]

The mIoU computed during training is an approximation -- it starts artificially high because it sets IoU for unseen classes = 100 to begin with and then updates with correct values as those classes are observed. Once the mIoU reaches a min value you can trust rises from that point onward, but the initial values in the first 1000 iterations are not meaningful and you should look at loss instead.

For the problem with not reaching > 60 accuracy. This depends on your discriminator initialization. One test to consider is to fix the representation and train only the discriminator for a couple thousand iterations and make sure you can get >60 accuracy. You may then use these discriminator weights as initialization for joint training of representation and discriminator. This was a technique used during initial experimentation, but was found to not be needed for the final paper model.

[Luodian]

Thank you~

[wang338]

I met the same problem, that when the iteration is about 3000,there is a error and stop the code.shows:"No suitable discriminator found -- returning." so how to solve this problem? have you ever met? thank you.

[wang338]

@jhoffman I encountered the same problem. the domacc value is set 60. But until the end of training, the discriminator network's accuracy can't reach that high.

@Luodian Hi, luodian. have you met the situation: the code was end with error:'No suitable discriminator found -- returning.' how do you solve it? thank you.

[Luodian]

@wang338 Usually in GTAV you will get a suitable D model after ~4000 iterations. It will jump out of training after 2950 iterations if you are using cityscapes as the target domain. If you don't get one D above 60 acc, then it breaks the training process. You just need to edit the break conditions like if iteration - last_update_g >= 3 * len(loader): at line 422 in train_fcn_adda.py. I obtain one D which performs about 75 acc after 8000 iterations. You can use a pre-trained model D when setting train_discrim_only=True or just let it first train D and waiting for D to achieve 60 acc, and then you will start to train your G for domain adaptation.

[wang338]

@wang338 Usually in GTAV you will get a suitable D model after ~4000 iterations. It will jump out of training after 2950 iterations if you are using cityscapes as the target domain. If you don't get one D above 60 acc, then it breaks the training process. You just need to edit the break conditions like if iteration - last_update_g >= 3 * len(loader): at line 422 in train_fcn_adda.py. I obtain one D which performs about 75 acc after 8000 iterations. You can use a pre-trained model D when setting train_discrim_only=True or just let it first train D and waiting for D to achieve 60 acc, and then you will start to train your G for domain adaptation.

ok, thank you so much. 爱你

[Luodian]

Hoping I clarified it clearly~If you have other questions probably we can go through QQ or Wechat.

[chep1126]

@Luodian Hi,are you using the original parameters provided by the auther? I obtain a poor result after 6700 iterations.And then it crashed due to "CUDA out of memory" .Would you tell me the configuration of your environment and the parameters? Thanks!

[ZHE-SAPI]

@Luodian 讲得挺清晰的，非常感谢！Clarified it pretty clear :-)， thank you so much !