search

Tsingularity / dift

[NeurIPS'23] Emergent Correspondence from Image Diffusion
https://diffusionfeatures.github.io
MIT License
594 stars 32 forks source link

Code about sparse feature matching #1

Closed shockwaveHe closed 1 year ago

shockwaveHe commented 1 year ago

Hi! Your work is amazing and I found that it may be helpful to some of my projects. I checked your paper and I am interested in DIFT sparse feature matching. It seems that your code doesn't include this part. Could you please share this code and exection tips? Thanks!

Tsingularity commented 1 year ago

Hi, thanks for your interest in our work!

I guess your are referring to the experiments on HPatches? If yes, for that part, we basically follow the practise of CAPS, where we first detect keypoints on both images using superpoint, then use DIFT to find mutual nearest neighbor matches, and lastly using cv2.findHomography() to remove the outliers and get estimated homography transformation. More details can be found in our paper or the section 4.2 of the CAPS paper.

Hope this helps!

Sjey-Lyn commented 7 months ago

Is there a reference code for this section please?

mayhemsloth commented 7 months ago

I'm also interested in DIFT as a potential sparse or dense geometric feature matching model to reliably produce mappings between a reference image and an initial image (as a part of a larger project). But it seems like based on your description of the practice in CAPS, that a sparse grid (much much less than every pixel) is produced on the reference image, and then the mapping is found for only those sparse points.

If I am more interested in a dense geometric feature matching model (that, say, takes in a reference and an initial image and outputs a per-pixel flow/correspondence/confidence mapping between these two images) would you recommend looking more into something like DKM or DiffMatch?