search

rpautrat / SuperPoint

Efficient neural feature detector and descriptor
MIT License
1.91k stars 418 forks source link

Ground Truth of Synthetic Ellipses #16

Closed mfaisal59 closed 6 years ago

mfaisal59 commented 6 years ago

Hi, In file synthetic_dataset.py line # 331 why the function is returning np.empty((0, 2), dtype=np.int), should it not return the centers of ellipses?

rpautrat commented 6 years ago

Hi, The ground truth on geometrical shapes are any corners of the shape. As there is no corner in an ellipse, it has no ground truth. The ellipses are here a kind of "counter-example", a shape with no ground truth. Note that we train here a corner detector, not a blob detector.

mfaisal59 commented 6 years ago

But in the paper (section 4.1) authors have stated that the ground truths are Y-junctions, L-junctions, T-junctions as well as centers of tiny ellipses and end points of line segments.

screenshot_1

rpautrat commented 6 years ago

Indeed, but I think that by "tiny ellipses" they meant "almost points". Here we deal with large ellipses. But I concede that we have no such tiny ellipses in the current dataset.

However, when you look at the sample of synthetic shapes in the appendix, they have no small ellipses, but only big ellipses with no ground truth. ellipses

mfaisal59 commented 6 years ago

Thanks for the clarification, I got the point.

ddetone commented 6 years ago

Hi, author of the SuperPoint paper here. I'd like to help clarify the confusion about the ellipses. Overall there were two ways we experimented with using them as ground truth.

One set of experiments was to create really narrow ellipses and treat the endpoints as ground truth, while leaving more circular shaped ellipses as negatives. These tended to help with repeatability, but only if you are really careful with the homographies apply on top of them. This is because when you apply more extreme homographies you can actually warp the narrow ellipses into circles and end up with inconsistent ground truth!

The other set of experiments we ran was to treat the centers of them as ground truth, and actually train a blob detector. This actually works! We trained a net on top of the blob ground truths but found it difficult at inference time to manage the scales of the network confidences across scales. More details can be found in Appendix Section B under "Blob Detection".

Hope this helps!

rpautrat commented 6 years ago

Thanks for the clarification @ddetone!