I performed self-supervised training of DINO on a pathology image dataset and analysed PCA as well as DINO-vit-feature.
At the same time, I also analysed the PCA in the model with DINO self-supervised learning in imagenet, and found that the colour images of the last 3dim of the PCA were more colourful in the model trained on pathology images, whereas the colour images in the model with DINO self-supervised learning in imagenet were were closer to the three primary colours of RGB and less colourful. What do you think these differences due to PCA represent?
I performed self-supervised training of DINO on a pathology image dataset and analysed PCA as well as DINO-vit-feature.
At the same time, I also analysed the PCA in the model with DINO self-supervised learning in imagenet, and found that the colour images of the last 3dim of the PCA were more colourful in the model trained on pathology images, whereas the colour images in the model with DINO self-supervised learning in imagenet were were closer to the three primary colours of RGB and less colourful. What do you think these differences due to PCA represent?
DINO:pretrained with pathology data
DINO:pretrained with imagenet data