search

mihaidusmanu / d2-net

D2-Net: A Trainable CNN for Joint Description and Detection of Local Features
Other
764 stars 163 forks source link

Could D2-Net learn features that are rotation invariant? #44

Closed riven314 closed 4 years ago

riven314 commented 4 years ago

Thanks for the great work! I would like to train d2-net for a problem but the problem involves image pairs with rotation transformation.

(Additionally, the images are actually karyotype image. The image pair contains a large raw karyotype image with a lot of overlapping chromosomes, and a small image with one chromosome instance perfectly cropped i.e its background removed. Also, two images have the same scale)

Do you have any insight if D2-Net is a good solution to this? Thank you!

mihaidusmanu commented 4 years ago

Hello. While in theory, D2-Net could learn rotation invariant features, we never actually tried to enforce that in training - we mostly used D2-Net features on tourism images (which are generally upright). Since, in your case, rotational invariance is very important, you can try using image rotations as data augmentation or even rotation invariant convolutional filters (see https://arxiv.org/pdf/1604.06720.pdf + related work).

riven314 commented 4 years ago

Thanks for your suggestion!