Attempted to integrate into hloc

[catid]

Tried to get hloc to use RoMa as a matcher: https://github.com/catid/hloc

The results were not good:

My Aachen Day-Night v1.1 Submissions
Method | day | night
hloc defaults | 88.5 / 95.5 / 98.5 | - / - / -
roma | 87.1 / 94.8 / 97.7 | 66.5 / 89.0 / 96.3

Any suggestions for how to get good performance with RoMa for matching?

[Parskatt]

I've never tried roma on aachen. Did you use the outdoor model? I know loftr used some "dark aug" for the nighttime scenes.

[catid]

Yeah using outdoor model:

    "roma": {
        "output": "matches-roma",
        "model": {
            "name": "roma",
            "weights": "outdoor",
            "max_keypoints": 2000,
            "match_threshold": 0.2,
        },
        "preprocessing": {
            "grayscale": False,
            "force_resize": True,
            "resize_max": 1024,
            "width": 320,
            "height": 240,
            "dfactor": 8,
        },
        'max_error': 2,  # max error for assigned keypoints (in px)
        'cell_size': 8,  # size of quantization patch (max 1 kp/patch)
    },

[catid]

Maybe something related to the max_keypoints since RoMa produces a ton of points, or perhaps the preprocessing width/height/dfactor stuff is wrong.. seems to expect w,h=672?

[Parskatt]

Its very hard to debug without runninf it myself, but it should be almost impossible to fail at aachen. Some possible things are,

1. Images are preprocessed wrong. They seem to be small size? Are they standardized with imagenet values?
2. Too many matches from false positives.
3. Other

My suggestion would be, try to debug what comes in and what comes out of roma. Save some images to disk, with both the roma samples matches and the warp and confidence as demonstrated in our demo.

[Parskatt]

Maybe something related to the max_keypoints since RoMa produces a ton of points, or perhaps the preprocessing width/height/dfactor stuff is wrong.. seems to expect w,h=672?

672 is what we used for best results (including upsampling which requires original images, as in pdcnet), but you should be fine with 560. 320 seems much too low, but I'm not even sure how you got that to run? RoMa requires image sizes to be multiples of 14.

[Parskatt]

In any case, I'm happy to see you try it out on Aachen, and happy to help debug if something further comes up.

[Parskatt]

There seems to be a bug in match dense in hloc, see this line:

https://github.com/catid/hloc/blob/4370642b4417261b7dddfb46532626d252ec23f0/hloc/match_dense.py#L167

Perhaps that is causing issues?

See also: https://github.com/cvg/Hierarchical-Localization/issues/303

[catid]

Oh haha yeah that could cause problems! Fixed in my branch and now waiting 8 hours for the results :D

[catid]

SPSG | 90.0 / 96.4 / 99.4 | 77.5 / 90.6 / 100.0

Without fix:

roma | 87.1 / 94.8 / 97.7 | 66.5 / 89.0 / 96.3

With fix:

roma | 87.1 / 94.4 / 97.8 | 69.6 / 89.0 / 97.9

[Parskatt]

My best guesses from looking through the code without running are,

1. 2000 matches might be too low to get good keypoints with roma, it might work better to assign to superpoint.

2. The 0.2 threshold doesnt actually prevent matches under 0.2, it just puts all conf above it to 1. This means that even for non matching pairs we will always get 2000 matches. Probably this huge amount of outlier matches can cause issues.

[sarlinpe]

Bug fixed in https://github.com/cvg/Hierarchical-Localization/commit/c3479fd57045a80b573d471be6febfc2873d207d, thanks for finding and sorry about that.

[hit2sjtu]

Any update on RoMa on Aachen? Interested in the results. Thanks! @catid

[Parskatt]

Hi, I've got RoMa to produce good results on Aachen by matching superpoint keypoints (basically replacing superglue). These are the results I get:

At a later point, I'll release some code for reproducing these results.

[HtutLynn]

@Parskatt , Hi, I'm very interested in the impressive results RoMa has achieved on the Aachen dataset using SuperPoint keypoints. May I know the estimated timeline for when you might release the code for reproducing these results?