search

jhoffman / cycada_release

Code to accompany ICML 2018 paper
BSD 2-Clause "Simplified" License
561 stars 126 forks source link

Understanding about semantic consistency loss #21

Closed John1231983 closed 4 years ago

John1231983 commented 5 years ago

Thanks for sharing a good work. I have a question related to semantic consistency loss (Eq. 4)

You first train a source task model fs (it is similar to training a segmentation network with input and label (Xs, Ys)). Then you will freeze the model fs. In the second phase, you feed the Xt to fs to obtain fs(Xt)-->arg to obtain label, and feed the synthetic target image fs(G(Xt)). You use cross-entropy to compute the task loss

My question is that the G(Xt) is the synthetic of the source domain. fs(G(Xt)) will work because fs is trained on source domain. However, the Xt is the target domain, fs(Xt) will never work or noise in the segmentation result. Why do you use the fs(Xt) as the label to compute cross entropy?

jianingwangind commented 5 years ago

@John1231983 I'm not sure i'm getting you right. I think it doesn't matter, whether the results coming from fs is accurate or not. What we want is that the output of fs should be the same before and after translation, even the output is incorrect. I hope it can help you.

John1231983 commented 5 years ago

@jianingwangind : Thanks. I came back after the holiday.

I think the problem is that the two domains are different. So, fs is trained in the source domain cannot provide good results on the target domain. So, the loss is very high, I guess.

jhoffman commented 4 years ago

The assumption is that the source classifier has better than chance performance on the target date so that f_s(x_t) had some semantic meaning (of course not perfect or no need to adapt!). As said above the loss asks that the semantic label (whatever it is) remains unchanged before and after image translation.