Why is there no forward direction process in BDL.py ?

[nakashima-kodai]

In your paper, using the bidirectional lerning, the image translation model and the segmentation adaptation model can be learned alternatively. But, I think there is not forward direction process in BDL.py. Please teach me the reason. Thank you in advance.

[liyunsheng13]

I think you mean the image translation model. I put it in the folder named cyclegan. But I only upload the different files compared to the official CycleGAN model.

[nakashima-kodai]

What I want to say is whether it is acceptable not to learn image translation model and segmentation model at the same time.

[liyunsheng13]

Yes, you can

[nakashima-kodai]

How can I do that ?

[liyunsheng13]

It seems that I don’t fully understand you question. You can train the BDL.py to learn segmentation model and train cyclegan to learn the image translation model. Is that clear?

[nakashima-kodai]

Yes, I know that. To sum up, does it mean that it should be executed like train_cyclegan.py (the result is uploaded)-> BDL.py-> SSL.py-> BDL.py?

[liyunsheng13]

Yes, I also upload the model to train SSL.py.

[nakashima-kodai]

Oh, ok. Thank you very much.

[liyunsheng13]

No problem

[DLwbm123]

Hi! I did't get the idea clearly either. There is really no training code for cycleGAN in BDL.py. And cycleGAN need to be trained first so that it didn't get feedback from segmentation model to update. So, could you explain more accurately and clearly about the ''Bidirectional training'' process? In my mind, the cycleGAN should be learned with the segmentation model in an alternative way in BDL.py instead of training it separately before training segmentation model. Thank you!

[liyunsheng13]

First of all, it is not an end-to-end system. I train CycleGAN and Segmentation model separately. Actually, if you combine them together, you will have the memory issue and also the unstable training process. What I did here is to train a CycleGAN first (code is in the fold of CycleGAN) and then use the translated images to train the segmentation model with BDL.py. In order to build feedback between the two models to implement the bidirectional learning, you can use the segmentation model as a perception loss to help train the CycleGAN again. The whole code for CycleGAN still needs sometime to be cleaned but currently I upload the most important part of it (I upload the different files compared to the official CycleGAN work). I also upload the translated images given by CycleGAN, you can easily reproduce the results in the paper by using those images with BDL.py. Maybe the name BDL.py confuses you a little bit, I will rename it with 'segmentation.py' or other names. Hope my explanation helps you

[DLwbm123]

First of all, it is not an end-to-end system. I train CycleGAN and Segmentation model separately. Actually, if you combine them together, you will have the memory issue and also the unstable training process. What I did here is to train a CycleGAN first (code is in the fold of CycleGAN) and then use the translated images to train the segmentation model with BDL.py. In order to build feedback between the two models to implement the bidirectional learning, you can use the segmentation model as a perception loss to help train the CycleGAN again. The whole code for CycleGAN still needs sometime to be cleaned but currently I upload the most important part of it (I upload the different files compared to the official CycleGAN work). I also upload the translated images given by CycleGAN, you can easily reproduce the results in the paper by using those images with BDL.py. Maybe the name BDL.py confuses you a little bit, I will rename it with 'segmentation.py' or other names. Hope my explanation helps you

I get more clear about the idea and I will read the codes more carefully. Thanks for you kindly explanation and patience!

[changliu816]

Hi! For the translated GTA5 as Cityscape dataset by CycleGAN you uploaded, it is the result of first-time trained CycleGAN or the second time trained CycleGAN with perception loss ? Thank you !

[liyunsheng13]

It is the translated images given by the second CycleGAN with perception loss.

[ghost]

Hi,Can you tell me the value of finesize and loadsize when testing? @liyunsheng13

[liyunsheng13]

The loadsize is 1024 and the finesize is 452 when I did experiments. The gpu I use in Microsoft has 16G memory. So if the gpu you used only has 12G memory, I suggest you to decrease the finesize to 400 or some other values, otherwise it will have the out of memory issue. Although I think decreasing the finesize might has a bad influence on the performance, the influence might be very tiny.

[ghost]

The transferred image you provided is 1024x568. Is this because you get the 452X452 image and resize it to 1024X568?so 1024X568 is the input size of segmentation model and dont need crop to 321 or some other values?@liyunsheng13

[liyunsheng13]

No, I resize the image to 1024568 and crop to 452452 to train cyclegan. I suggest you to read the code or the paper more carefully.

[ghost]

ok,thank you very much

[octpath]

What I did here is to train a CycleGAN first (code is in the fold of CycleGAN) and then use the translated images to train the segmentation model with BDL.py. In order to build feedback between the two models to implement the bidirectional learning, you can use the segmentation model as a perception loss to help train the CycleGAN again.

Hi. I have some questions about initial weights of CycleGAN. When I train CycleGAN with perception loss for the first time, are the initial weights of the generators and the discriminators transfered from vanilla (without perception loss) CycleGAN? Also, are init weights of 2nd CycleGAN w/ perception loss transfered from 1st one? Thank you.

[liyunsheng13]

For training CycleGAN, you can just randomly initialize it for both the 1st and the 2nd time.