rpautrat / SuperPoint

Efficient neural feature detector and descriptor
MIT License
1.88k stars 416 forks source link

Boundary checking on bordering artifacts #264

Closed toru34 closed 2 years ago

toru34 commented 2 years ago

To pick up candidates which don't produce bordering artifacts, you write this way (link)

valid = tf.where(tf.reduce_all((scaled >= 0.) & (scaled < 1.), [1, 2]))[:, 0]

Why do you set scaled < 1, not scaled <= 1? It would be more reasonable to use <= for coordinate values.

toru34 commented 2 years ago

Same for rotation.

rpautrat commented 2 years ago

These are not integer values, but float ones. So using '<' or '<=' should not make any difference. The probability to get a scale that will get one coordinate to be exactly 1 is 0.

toru34 commented 2 years ago

Here is your code.

scales = tf.concat(
        [[1.], tf.truncated_normal([n_scales], 1, scaling_amplitude/2)], 0)
center = tf.reduce_mean(pts2, axis=0, keepdims=True)
scaled = tf.expand_dims(pts2 - center, axis=0) * tf.expand_dims(
        tf.expand_dims(scales, 1), 1) + center
if allow_artifacts:
    valid = tf.range(1, n_scales + 1)  # all scales are valid except scale=1
else:
    valid = tf.where(tf.reduce_all((scaled >= 0.) & (scaled < 1.), [1, 2]))[:, 0]

If allow_artifacts=False、valid should include scale=1 (without any scaling) at index=0. But the code above rejects this scale=1 index, since this uses <. So it needs to use <= instead of <. Actually if all the random numbers were not valid in tf.where condition in addition to scale=1 (because of using <), it would reject all the indices, ending up an error.

rpautrat commented 2 years ago

Yes, but because of the patch ratio < 1 normally, the scaled points are never going up to 1 when scale=1. So using < 1 should still be fine without using artifacts.

Nevertheless, I updated the code with <=, just to avoid any crash if someone uses a patch ratio of 1.

Thanks for reporting this.