Use of projection head

[ekorman]

If I understand the code correctly, then the experiments with projection heads (e.g. script/run_contrastive_da.sh) use the projection head at test time and not just training time. This is from looking at resnet_util.ResNet which looks like the output embeds is always the projection head. So, in the notation of the paper (e.g. figure 4), the contrastive results generated by this repo are all Contrastive (DA) g o f

So then in table 2 of the paper where it says Contrastive (DA) that refers to using g o f`` since the scriptscript/run_contrastive_da.sh` is supposed to reproduce that row?

That's a little confusing to me since table 4 shows that the best results are obtained without using the projection head at test time (i.e. in the second stage).

Is my understanding correct?

[kihyuks]

Hi,

https://github.com/google-research/deep_representation_one_class/blob/a36feaa2e2f1f8d4aabe24d219e2220b187bd315/util/train.py#L586-L592

As shown above, we extract multiple representations for evaluation:

1. logits: embedding after projection head and classifier weights (only applicable for RotNet)
2. embeds (g o f): embedding after projection head
3. pools (f): embedding before projection head

As mentioned in Section 3.1, we use f as an embedding for test time unless otherwise specified as in Figure 4.

[ekorman]

ah got it, thanks for the quick response!