search

mihaidusmanu / d2-net

D2-Net: A Trainable CNN for Joint Description and Detection of Local Features
Other
761 stars 163 forks source link

D2Net to kornia #84

Open ducha-aiki opened 3 years ago

ducha-aiki commented 3 years ago

Hi,

Would you be interested for porting the inference part of the D2Net into the kornia? The idea is to allow simple usage of local features, like this:

local_feat = kornia.feature.D2Net(pretrained=True)
lafs, resps, descs = local_feat(image)

We now have HardNet, SOSNet, TFeat and other descriptors. It may also increase the usage of D2Net :)

mihaidusmanu commented 3 years ago

Hey. Thanks for reaching out - sounds like a neat idea! I was planning to work on a cleaner implementation at some point, so why not integrate it with Kornia.

If I understand it correctly, I simply need to create a new D2Net nn.Module similar to e.g. the HardNet one.

I'll give it a try in the following weeks and reach out with a PR when it's ready!

ducha-aiki commented 3 years ago

Sounds great, thank you!

A couple of things regarding the format implementation in a separate file. It is good to follow the output format of the https://kornia.readthedocs.io/en/latest/feature.html#kornia.feature.ScaleSpaceDetector:

with the exception that descriptors are also returned, so 3 outputs.

To create LAFs (kornia format for the local features), you can use function https://kornia.readthedocs.io/en/latest/feature.html#kornia.feature.laf_from_center_scale_ori

where you feed all zeros instead of orientation and D2Net receptive field as a scale parameter.

ducha-aiki commented 2 years ago

Hi Mihai,

Did you have a chance to think about this, or do you need any help?