search

TruongKhang / DeViLoc

[CVPR2024 Oral] Learning to Produce Semi-dense Correspondences for Visual Localization
Apache License 2.0
87 stars 5 forks source link

results on Cambridge and 7Scenes #1

Closed feixue94 closed 4 months ago

feixue94 commented 4 months ago

Dear author,

Excellent work. I am wondering if you could also provide the evaluation results on 7Scenes and Cambridge landmarks with the metric like success ratio with errors smaller than (5cm, 5◦) as shown in PRAM (https://arxiv.org/pdf/2404.07785.pdf).

I will be very happy to put your results in the refined version of PRAM.

Best,

TruongKhang commented 4 months ago

Hi @feixue94 , please find the results below. I just copied the full logs for you. Feel free to get the results of any metrics you want.

7Scenes:

Percentage of test images localized within:
    1cm, 1deg : 10.35%
    2cm, 2deg : 37.25%
    3cm, 3deg : 66.30%
    5cm, 5deg : 92.55%
    25cm, 2deg : 98.70%
    50cm, 5deg : 100.00%
    500cm, 10deg : 100.00%
Creating the reference model.
Kept 2000 images out of 4000.
2023-11-20 16:11:39.297 | INFO     | __main__:<module>:293 - DataLoader of scene fire is initialized!
[2023/11/20 16:11:53 hloc.pipelines.Cambridge.utils INFO] Results for file fire_poses.txt:
Median errors: 0.017m, 0.744deg
Percentage of test images localized within:
    1cm, 1deg : 16.70%
    2cm, 2deg : 60.00%
    3cm, 3deg : 83.05%
    5cm, 5deg : 95.55%
    25cm, 2deg : 96.25%
    50cm, 5deg : 99.85%
    500cm, 10deg : 99.85%
Creating the reference model.
Kept 1000 images out of 2000.
2023-11-20 16:12:12.354 | INFO     | __main__:<module>:293 - DataLoader of scene heads is initialized!
[2023/11/20 16:12:14 hloc.pipelines.Cambridge.utils INFO] Results for file heads_poses.txt:
Median errors: 0.009m, 0.650deg
Percentage of test images localized within:
    1cm, 1deg : 53.50%
    2cm, 2deg : 88.60%
    3cm, 3deg : 92.50%
    5cm, 5deg : 97.70%
    25cm, 2deg : 96.90%
    50cm, 5deg : 98.20%
    500cm, 10deg : 99.20%
Creating the reference model.
Kept 6000 images out of 10000.
2023-11-20 16:12:54.603 | INFO     | __main__:<module>:293 - DataLoader of scene office is initialized!
[2023/11/20 16:13:06 hloc.pipelines.Cambridge.utils INFO] Results for file office_poses.txt:
Median errors: 0.027m, 0.815deg
Percentage of test images localized within:
    1cm, 1deg : 7.65%
    2cm, 2deg : 32.82%
    3cm, 3deg : 56.62%
    5cm, 5deg : 82.93%
    25cm, 2deg : 94.90%
    50cm, 5deg : 100.00%
    500cm, 10deg : 100.00%
Creating the reference model.
Kept 4000 images out of 6000.
2023-11-20 16:13:32.155 | INFO     | __main__:<module>:293 - DataLoader of scene pumpkin is initialized!
[2023/11/20 16:13:39 hloc.pipelines.Cambridge.utils INFO] Results for file pumpkin_poses.txt:
Median errors: 0.038m, 1.025deg
Percentage of test images localized within:
    1cm, 1deg : 5.55%
    2cm, 2deg : 21.65%
    3cm, 3deg : 37.90%
    5cm, 5deg : 62.70%
    25cm, 2deg : 76.40%
    50cm, 5deg : 98.30%
    500cm, 10deg : 100.00%
Creating the reference model.
Kept 7000 images out of 12000.
2023-11-20 16:14:42.766 | INFO     | __main__:<module>:293 - DataLoader of scene redkitchen is initialized!
[2023/11/20 16:15:05 hloc.pipelines.Cambridge.utils INFO] Results for file redkitchen_poses.txt:
Median errors: 0.034m, 1.190deg
Percentage of test images localized within:
    1cm, 1deg : 3.64%
    2cm, 2deg : 19.64%
    3cm, 3deg : 41.16%
    5cm, 5deg : 72.92%
    25cm, 2deg : 82.20%
    50cm, 5deg : 99.18%
    500cm, 10deg : 99.90%
Creating the reference model.
Kept 2000 images out of 3000.
2023-11-20 16:15:18.985 | INFO     | __main__:<module>:293 - DataLoader of scene stairs is initialized!
[2023/11/20 16:15:22 hloc.pipelines.Cambridge.utils INFO] Results for file stairs_poses.txt:
Median errors: 0.042m, 1.118deg
Percentage of test images localized within:
    1cm, 1deg : 2.30%
    2cm, 2deg : 20.10%
    3cm, 3deg : 34.90%
    5cm, 5deg : 60.50%
    25cm, 2deg : 78.10%
    50cm, 5deg : 91.10%
    500cm, 10deg : 98.20%

Cambridge:

2023-11-20 20:05:58.852 | INFO     | __main__:<module>:293 - DataLoader of scene ShopFacade is initialized!
[2023/11/20 20:05:58 hloc.pipelines.Cambridge.utils INFO] Results for file ShopFacade_poses.txt:
Median errors: 0.040m, 0.182deg
Percentage of test images localized within:
    1cm, 1deg : 1.94%
    2cm, 2deg : 16.50%
    3cm, 3deg : 34.95%
    5cm, 5deg : 59.22%
    25cm, 2deg : 94.17%
    50cm, 5deg : 99.03%
    500cm, 10deg : 100.00%
[2023/11/20 20:06:03 hloc.pipelines.Cambridge.utils INFO] Scaling the COLMAP model to the original image size.
2023-11-20 20:06:46.482 | INFO     | __main__:<module>:293 - DataLoader of scene OldHospital is initialized!
[2023/11/20 20:06:46 hloc.pipelines.Cambridge.utils INFO] Results for file OldHospital_poses.txt:
Median errors: 0.131m, 0.283deg
Percentage of test images localized within:
    1cm, 1deg : 0.00%
    2cm, 2deg : 0.55%
    3cm, 3deg : 1.65%
    5cm, 5deg : 12.64%
    25cm, 2deg : 69.23%
    50cm, 5deg : 87.36%
    500cm, 10deg : 100.00%
[2023/11/20 20:07:01 hloc.pipelines.Cambridge.utils INFO] Scaling the COLMAP model to the original image size.
2023-11-20 20:07:54.634 | INFO     | __main__:<module>:293 - DataLoader of scene GreatCourt is initialized!
[2023/11/20 20:07:54 hloc.pipelines.Cambridge.utils INFO] Results for file GreatCourt_poses.txt:
Median errors: 0.178m, 0.109deg
Percentage of test images localized within:
    1cm, 1deg : 0.39%
    2cm, 2deg : 2.50%
    3cm, 3deg : 5.53%
    5cm, 5deg : 10.26%
    25cm, 2deg : 62.11%
    50cm, 5deg : 80.00%
    500cm, 10deg : 98.16%
[2023/11/20 20:08:03 hloc.pipelines.Cambridge.utils INFO] Scaling the COLMAP model to the original image size.
2023-11-20 20:09:01.894 | INFO     | __main__:<module>:293 - DataLoader of scene KingsCollege is initialized!
[2023/11/20 20:09:03 hloc.pipelines.Cambridge.utils INFO] Results for file KingsCollege_poses.txt:
Median errors: 0.124m, 0.206deg
Percentage of test images localized within:
    1cm, 1deg : 0.00%
    2cm, 2deg : 1.17%
    3cm, 3deg : 3.50%
    5cm, 5deg : 12.83%
    25cm, 2deg : 78.43%
    50cm, 5deg : 93.29%
    500cm, 10deg : 100.00%
[2023/11/20 20:09:18 hloc.pipelines.Cambridge.utils INFO] Scaling the COLMAP model to the original image size.
2023-11-20 20:10:27.900 | INFO     | __main__:<module>:293 - DataLoader of scene StMarysChurch is initialized!
[2023/11/20 20:10:28 hloc.pipelines.Cambridge.utils INFO] Results for file StMarysChurch_poses.txt:
Median errors: 0.074m, 0.232deg
Percentage of test images localized within:
    1cm, 1deg : 0.75%
    2cm, 2deg : 5.28%
    3cm, 3deg : 13.02%
    5cm, 5deg : 30.38%
    25cm, 2deg : 96.04%
    50cm, 5deg : 99.62%
    500cm, 10deg : 99.81%
feixue94 commented 4 months ago

Great. Thank you so much.