I wanted to share my observations regarding the correct usage of moco v3 as a backbone, as reported by the moco official repository, as well as a potential inconsistency in the repository.
In the file main_lincls.py of the moco v3 repository, the module self.base_encoder.head is removed assuming that ViT is being used as the backbone. However, in the file get_features_mocov3.py of the same repository, the fc layer self.base_encoder is still present in the backbone, which should be nn.Identity() in this case, as did to Ctranspath.
I just wanted to share my observations and this does not indicate any doubts towards the experiment results. In fact, I really enjoyed this work a lot.
Heyy,
I wanted to share my observations regarding the correct usage of moco v3 as a backbone, as reported by the moco official repository, as well as a potential inconsistency in the repository.
In the file main_lincls.py of the moco v3 repository, the module self.base_encoder.head is removed assuming that ViT is being used as the backbone. However, in the file get_features_mocov3.py of the same repository, the fc layer self.base_encoder is still present in the backbone, which should be nn.Identity() in this case, as did to Ctranspath.
I just wanted to share my observations and this does not indicate any doubts towards the experiment results. In fact, I really enjoyed this work a lot.
I am open to have a discussion with followers.