search

facebookresearch / dinov2

PyTorch code and models for the DINOv2 self-supervised learning method.
Apache License 2.0
8.3k stars 699 forks source link

"inverted" first principal component of vitb14_reg4 #420

Closed amundra15 closed 1 month ago

amundra15 commented 1 month ago

I tried using the vitb model with registers, and it seems like the first principal component behaves differently from the model without registers.

Input RGB: input(1)

PCA(n=1) of patch tokens of vitb14: dino_fgbg

PCA(n=1) of patch tokens of vitb14_reg4: dino_reg4_fgbg

The values are overlayed in red. (ignore the mismatch in the image orientation).

Since this is an emergent behaviour, I don't think it is a problem in itself. But I want to confirm if others have also observed the same behaviour.

TheoMoutakanni commented 1 month ago

Hello @amundra15, This is an expected behaviour of the PCA. The sign of the components can be flipped without changing the decomposition. You just need to be aware of this phenomenon.