How to use MoCo v3 with custom model (3D ViT)

[hiroalchem]

I am a lightly (and self-supervised) novice and I am trying to use MoCo v3, but I could only find MoCo v2 as an example. However, I could only find MoCo v2 as an example of its use. Also, are there any examples of how to apply it to custom models (especially ones like 3D ViT)?

[guarin]

Hi, we currently don't have examples for MoCo v3. But you can follow the v2 example and set the MoCoProjectionHead parameters as in the paper. You'll also have to add a prediction head, for this you can also use MoCoProjectionHead and adapt parameters. Then set memory queue length to 0 as MoCo v3 doesn't use a queue anymore (if you have large batch size >=4'096).

You can use vit backbone as follows:

from torch import nn
vit = torchvision.models.vit_l_16(pretrained=False)
vit.heads = nn.Identity() # remove classification head

And then use vit as backbone in moco. I didn't test this, so you might have to adapt some things, but I hope this helps :)

[guarin]

Closing due to no activity