Open josephcappadona opened 2 months ago
For a SSL model like a ViT with DINO and iBOT heads, where should the MuReadout be placed? At the output of the backbone, or the output of the projection heads, or both?
Has anyone used muP with DINOv2 before? I did some searching but found nothing.
For a SSL model like a ViT with DINO and iBOT heads, where should the MuReadout be placed? At the output of the backbone, or the output of the projection heads, or both?
Has anyone used muP with DINOv2 before? I did some searching but found nothing.