Open AHammoudeh opened 1 year ago
+1
Any updates on how to compute the precomputed foreground / background projection?
+1
any update on this question?
+1
I'm not one of the authors but I assume that they took the PCA of a bunch of images and then took the vector of the first component (pca.components_[0]
). I'd spot check if this vector points towards the background objects, and if so, multiply it by -1
to make it point towards the foreground objects so that the thresholding works as expected.
Having this pre-computed vector is nice because 1) it makes a cleaner foreground/background separation, and 2) it's cheaper to take a dot product of your patch embedding than to do two rounds of training PCA.
I'm not one of the authors but I assume that they took the PCA of a bunch of images and then took the vector of the first component (
pca.components_[0]
). I'd spot check if this vector points towards the background objects, and if so, multiply it by-1
to make it point towards the foreground objects so that the thresholding works as expected.Having this pre-computed vector is nice because 1) it makes a cleaner foreground/background separation, and 2) it's cheaper to take a dot product of your patch embedding than to do two rounds of training PCA.
I think this is right. And the images selected to calculate the vector(PCA analysis) will actually determine the mask.
Regarding the code below from the features notebook, the standard array, which was used to separate foreground/background, was loaded from a numpy file saved on the web. Can someone please advise how that standard array can be computed? It consists of a vector of 768 numbers.
``
Precomputed foreground / background projection
STANDARD_ARRAY_URL = "https://dl.fbaipublicfiles.com/dinov2/arrays/standard.npy" standard_array = load_array_from_url(STANDARD_ARRAY_URL)