with perm_neg in line 383 one random image tensor of the batch gets chosen. (or its embeddings)
Then samples are created from the original and the random chosen. But only from the batch?
coords2 are coordinates that will get used for grid_sample that transforms the chosen image tensor?
afterwards the grid sample from the frozen backbone and the head will get corrolated
so I'm not sure if i don't quite grasp the code yet or this code just gets a random image from current batch?
that meaning, a big batch size is quite important to get enough negative corrolation?
anyone can help me with this one, or even @mhamilton723 ?
Hi guys,
this is a fantastic approach!
I understand how the self corrolation and the kNN corrolation is computed but i dont quite get how the random image corrolation is computed.
https://github.com/mhamilton723/STEGO/blob/eb4d6b521740bd4265681b353547f2ffca65d673/src/modules.py#L382-L391
with
perm_negin line 383 one random image tensor of the batch gets chosen. (or its embeddings)Then samples are created from the original and the random chosen. But only from the batch?
coords2are coordinates that will get used for grid_sample that transforms the chosen image tensor?afterwards the grid sample from the frozen backbone and the head will get corrolated
so I'm not sure if i don't quite grasp the code yet or this code just gets a random image from current batch?
that meaning, a big batch size is quite important to get enough negative corrolation?
anyone can help me with this one, or even @mhamilton723 ?
Thanks in advance