Keypoints Pixels & Heatmap

[tolboothtimemachine]

I'd love to be able to generate a heatmap to overlay on each of the images used in a comparison to visualize the distribution of keypoints and highlight significant areas.

I don't have much experience handling torch.Tensor outputs, but I see the data below for an image comparison

kpts0[:3] - tensor([[1012.0620, 455.7500], [1371.6401, 755.7500], [1226.8101, 123.2500])

kpts1[:3] - tensor([[ 382.5601, 11.8231], [ 412.9731, 11.8231], [ 502.7641, 11.8231])

matches[:3] - tensor([[ 0, 794], [ 1, 1017], [ 2, 363])

Can you suggest the best way to return (x,y) pixel values of keypoints for each image?

Thanks! This model & paper are really excellent. :)

[Phil26AT]

Hi @tolboothtimemachine

To obtain detection heatmaps you need to make slight changes to the extractors. For SuperPoint, you can just add the score map in this line to the return values. It has shape [1 x h x w], so the same shape as your image (unless it was resized in extract). The scores are in the range [0, 1], where 1 marks a very interesting point.

Thank you for your interest in LightGlue :)