Q: Effect of multi-level features.

[msseibel]

Hi, thanks for your great work.

Table 9 states, that applying the contrastive loss to the deepest layer (4th layer) brings the largest improvement. However, applying it to layer 1 usually helped more than applying it to layer 3. That is a very interesting observation.

Further, you presented a t-SNE plot for the target domain, and I guess that the different colors refer to different classes. Did you also compute a t-SNE plot to examine whether the domains are sufficiently mixed? (I can't find a supplementary to the paper.)

Perhaps a t-SNE analysis could guide us to decide where to apply the contrastive loss? Perhaps it also depends on the progress of the training?

Best, Marc

[BinhuiXie]

Hi @msseibel

thanks for your attention.

actually, for CNN-based architecture (DeepLabv2), the best way is to apply contrastive loss to the last layer. for the second best result is obtained when applied to the first layer, we guess it might allow detailed object boundary recovery, which is consistent with partial design of DeepLabV3+.

yep, the colors refer to different classes and are consistent with qualitative results in Fig. 4.

we haven't examined the whole domain discrepancy with t-SNE. If interested, the code could be found at our another repo

it might be inspiring to plot t-SNE w.r.t. training progress.