search

microsoft / lamar-benchmark

Source code for the ECCV 2022 paper "Benchmarking Localization and Mapping for Augmented Reality".
Creative Commons Attribution 4.0 International
384 stars 36 forks source link

which trajectories.txt is the true GT pose with highest accuracy? #78

Open eleboss opened 1 month ago

eleboss commented 1 month ago

Hi,

Thanks for the amazing benchmark.

I am confused with the trajectory.txt file in Lamar benchmark. In the data web server, there are two folders.

**1. benchmark,

  1. raw**

Inside the /raw there are multiple types of trajectories.txt like: ./raw/CAB/sessions/hl_2022-06-30-23-54-19-330/trajectories.txt:

# camera-to-world
97058366, hetrig, 0.9999999863068751, 0.0001331232481771267, 2.947169959644033e-06, 9.82637486544622e-05, -5.172249111253427e-05, -6.55288022711177e-05, -6.260384371584385e-05

./lamar/raw/CAB/sessions/ios_2021-06-03_12.16.58/trajectories.txt:

# timestamp, device_id, qw, qx, qy, qz, tx, ty, tz, *covar
675507361498, cam_phone_675507361498, -0.5487487879747363, -0.822305059439159, -0.004226377112917012, 0.15056989956003294, 0.0, 0.0, 0.0

./lamar/raw/CAB/sessions/navvis_2022-06-21_09.28.22/trajectories.txt:

# timestamp, device_id, qw, qx, qy, qz, tx, ty, tz, *covar
1655796529349653, cam0_center, 0.7268834635837101, -0.5426432763924871, -0.27288459263844134, 0.3204882276389068, -0.12168281123920213, -0.05460203925525969, 0.9820001913253169

Is the IOS one a relative pose in the submap? Is HL a global GT obtained using superpoint+superglue? Is navvis a globally aligned with PGO and BA optimized as mentioned in the ECCV paper?

I need to use the GT pose for some rendering tasks, so the accuracy is my major concern. It would be great if you could clarify some details.

Except that, I saw ./lamar/benchmark/CAB/sessions/query_val_hololens/proc/alignment_trajectories.txt. Is this traj an accurate gt pose, optimized with PGO and BA mentioned in the ECCV paper?

Thanks again.

Yours Shijie HKU

eleboss commented 1 month ago

I checked several git issues, https://github.com/microsoft/lamar-benchmark/issues/44 https://github.com/microsoft/lamar-benchmark/issues/10 https://github.com/microsoft/lamar-benchmark/issues/49

my best guess so far is : The IOS and HL traj.txt are both local poses generated by devices. They are not GT. Navvis's traj.txt is a globally GT (with PGO and BA optimized??). And it seems this is not directly connected with IOS and HL. So, I can use navvis's pointcloud, mesh, and traj.txt for visualization and other rendering process.

tianyilim commented 2 weeks ago

I think you're right. To get see aligned poses you would need to use one of their validation datasets eg. benchmark/CAB/sessions/query_val_phone/proc/alignment_trajectories/txt.