search

naver / kapture-localization

Provide mapping and localization pipelines based on kapture format
BSD 3-Clause "New" or "Revised" License
269 stars 42 forks source link

localize pipeline doesn't make trajectories.txt for localized result #20

Closed Keunmo closed 2 years ago

Keunmo commented 2 years ago

Environment:

python 3.8.10 kapture==1.1.5 kapture-localization==0.1.4

Problem:

kapture_pipeline_localize.py doesn't make trajectories.txt in kapture_localized_recover directory when I change query's record_data structure.

I followed the virtual_gallery_tutorial, and tried modify query kapture's record_data folder structure a little. This is original structure, image And This is virtual_gallery_tutorial_2, which I modified. image and I also modified records_camera.txt for virtual_gallery_tutorial2, like this. image

As shown in the picture, when I try kapture_pipeline_localize.py, virtual_gallery_tutorial_2 doesn't make trajectories.txt for query data.

Here is terminal output for virtual_gallery_tutorial, which is original version. 

$ kapture_pipeline_localize.py -v info \
      -i ./mapping \
      --query ./query \
      -kpt ./local_features/r2d2_500/keypoints \
      -desc ./local_features/r2d2_500/descriptors \
      -gfeat ./global_features/AP-GeM-LM18/global_features \
      -matches ./local_features/r2d2_500/NN_no_gv/matches \
      -matches-gv ./local_features/r2d2_500/NN_colmap_gv/matches \
      --colmap-map ./colmap-sfm/r2d2_500/AP-GeM-LM18_top5 \
      -o ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/ \
      --topk 5 \
      --config 2 \
> --skip evaluate export_LTVL2020
INFO    ::compute_image_pairs: compute_image_pairs. loading mapping: ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_mapping
INFO    ::compute_image_pairs: computing pairs with AP-GeM-LM18...
INFO    ::compute_image_pairs: compute_image_pairs. loading query: ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_query
INFO    ::compute_image_pairs: saving to file  ...
INFO    ::compute_image_pairs: all done
INFO    ::merge: Loading ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_mapping
INFO    ::merge: Loading ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_query
INFO    ::merge: Writing merged kapture data...
INFO    ::compute_matches: compute_matches. loading input: ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_map_plus_query
INFO    ::compute_matches: compute_matches. entering main loop...
100%|███████████████████████████████████████████████████████████████████████| 20/20 [00:00<00:00, 142.20it/s]
INFO    ::compute_matches: all done
INFO    ::run_colmap_gv: run_colmap_gv...
INFO    ::run_colmap_gv: remove rigs notation.
INFO    ::colmap: registering 6 sensors (cameras) in database...
INFO    ::colmap: registering 16 images in database...
INFO    ::colmap: registering 16 keypoints in database...
INFO    ::colmap: registering 20 matches in database...
INFO    ::colmap: ['colmap', 'matches_importer', '--database_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_map_plus_query_gv/colmap.db', '--match_list_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_map_plus_query_gv/match_list.txt', '--match_type', 'pairs', '--SiftMatching.use_gpu', '0']

==============================================================================
Custom feature matching
==============================================================================

Matching block [1/1] in 0.050s
Elapsed time: 0.001 [minutes]
16it [00:00, 14134.13it/s]
INFO    ::colmap: keeps 100.0% of verified matches (20/20) ...
100%|██████████████████████████████████████████████████████████████████████| 20/20 [00:00<00:00, 2282.12it/s]
INFO    ::colmap_localize: loading kapture files...
WARNING ::colmap_localize: Input data contains trajectories: they will be ignored
INFO    ::colmap_localize: remove rigs notation.
12it [00:00, 41665.27it/s]
INFO    ::colmap_localize: Step 1: Add precomputed keypoints and matches to colmap db
INFO    ::colmap_localize: Step 2: Run geometric verification - skipped
INFO    ::colmap_localize: Step 3: Run image_registrator
INFO    ::colmap: ['colmap', 'image_registrator', '--database_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/colmap.db', '--input_path', './colmap-sfm/r2d2_500/AP-GeM-LM18_top5/reconstruction', '--output_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/reconstruction', '--Mapper.ba_refine_focal_length', '0', '--Mapper.ba_refine_principal_point', '0', '--Mapper.ba_refine_extra_params', '0', '--Mapper.min_num_matches', '4', '--Mapper.init_min_num_inliers', '4', '--Mapper.abs_pose_min_num_inliers', '4', '--Mapper.abs_pose_min_inlier_ratio', '0.05', '--Mapper.ba_local_max_num_iterations', '50', '--Mapper.abs_pose_max_error', '20', '--Mapper.filter_max_reproj_error', '12']

==============================================================================
Loading database
==============================================================================

Loading cameras... 6 in 0.000s
Loading matches... 50 in 0.000s
Loading images... 16 in 0.001s (connected 16)
Building correspondence graph... in 0.002s (ignored 0)

Elapsed time: 0.000 [minutes]

==============================================================================
Registering image #13 (13)
==============================================================================

  => Image sees 79 / 96 points

Pose refinement report
----------------------
    Residuals : 176
   Parameters : 6
   Iterations : 10
         Time : 0.00658488 [s]
 Initial cost : 0.627017 [px]
   Final cost : 0.617635 [px]
  Termination : Convergence

==============================================================================
Registering image #14 (14)
==============================================================================

  => Image sees 84 / 101 points

Pose refinement report
----------------------
    Residuals : 166
   Parameters : 6
   Iterations : 14
         Time : 0.00102401 [s]
 Initial cost : 0.707709 [px]
   Final cost : 0.664657 [px]
  Termination : Convergence

==============================================================================
Registering image #15 (15)
==============================================================================

  => Image sees 274 / 366 points

Pose refinement report
----------------------
    Residuals : 616
   Parameters : 6
   Iterations : 14
         Time : 0.00341797 [s]
 Initial cost : 0.755861 [px]
   Final cost : 0.653651 [px]
  Termination : Convergence

==============================================================================
Registering image #16 (16)
==============================================================================

  => Image sees 304 / 353 points

Pose refinement report
----------------------
    Residuals : 668
   Parameters : 6
   Iterations : 7
         Time : 0.00177789 [s]
 Initial cost : 0.514305 [px]
   Final cost : 0.512528 [px]
  Termination : Convergence

INFO    ::colmap_localize: Step 4: Export reconstruction results to txt
INFO    ::colmap: ['colmap', 'model_converter', '--input_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/reconstruction', '--output_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/reconstruction', '--output_type', 'TXT']
INFO    ::colmap: importing colmap ...
INFO    ::colmap: parsing cameras  ...
16it [00:00, 15505.74it/s]
INFO    ::colmap: saving to kapture  ...
INFO    ::recover_timestamps_and_ids: recover_timestamps_and_ids...
INFO    ::recover_timestamps_and_ids: loading data ...
INFO    ::recover_timestamps_and_ids: recover records and trajectories
INFO    ::recover_timestamps_and_ids: recover sensor ids in sensors
INFO    ::recover_timestamps_and_ids: recover rig ids in rigs
INFO    ::recover_timestamps_and_ids: saving results
INFO    ::recover_timestamps_and_ids: handle image files with a call to transfer_actions
INFO    ::recover_timestamps_and_ids: import image files ...
INFO    ::recover_timestamps_and_ids: done.

And This is terminal output for virtual_gallery_tutorial_2, which is modified version. 

$ kapture_pipeline_localize.py -v info \
      -i ./mapping \
      --query ./query \
      -kpt ./local_features/r2d2_500/keypoints \
      -desc ./local_features/r2d2_500/descriptors \
      -gfeat ./global_features/AP-GeM-LM18/global_features \
      -matches ./local_features/r2d2_500/NN_no_gv/matches \
      -matches-gv ./local_features/r2d2_500/NN_colmap_gv/matches \
      --colmap-map ./colmap-sfm/r2d2_500/AP-GeM-LM18_top5 \
      -o ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/ \
      --topk 5 \
      --config 2 \
> --skip evaluate export_LTVL2020
INFO    ::compute_image_pairs: compute_image_pairs. loading mapping: ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_mapping
INFO    ::compute_image_pairs: computing pairs with AP-GeM-LM18...
INFO    ::compute_image_pairs: compute_image_pairs. loading query: ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_query
INFO    ::compute_image_pairs: saving to file  ...
INFO    ::compute_image_pairs: all done
INFO    ::merge: Loading ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_mapping
INFO    ::merge: Loading ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_query
INFO    ::merge: Writing merged kapture data...
INFO    ::compute_matches: compute_matches. loading input: ./colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_map_plus_query
INFO    ::compute_matches: compute_matches. entering main loop...
0it [00:00, ?it/s]
INFO    ::compute_matches: all done
INFO    ::run_colmap_gv: run_colmap_gv...
INFO    ::run_colmap_gv: remove rigs notation.
INFO    ::colmap: registering 6 sensors (cameras) in database...
INFO    ::colmap: registering 16 images in database...
INFO    ::colmap: registering 12 keypoints in database...
INFO    ::colmap: registering 0 matches in database...
INFO    ::colmap: ['colmap', 'matches_importer', '--database_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_map_plus_query_gv/colmap.db', '--match_list_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/kapture_inputs/proxy_map_plus_query_gv/match_list.txt', '--match_type', 'pairs', '--SiftMatching.use_gpu', '0']

==============================================================================
Custom feature matching
==============================================================================

Elapsed time: 0.000 [minutes]
16it [00:00, 12977.93it/s]
0it [00:00, ?it/s]
INFO    ::colmap_localize: loading kapture files...
WARNING ::colmap_localize: Input data contains trajectories: they will be ignored
INFO    ::colmap_localize: remove rigs notation.
12it [00:00, 100865.03it/s]
INFO    ::colmap_localize: Step 1: Add precomputed keypoints and matches to colmap db
INFO    ::colmap_localize: Step 2: Run geometric verification - skipped
INFO    ::colmap_localize: Step 3: Run image_registrator
INFO    ::colmap: ['colmap', 'image_registrator', '--database_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/colmap.db', '--input_path', './colmap-sfm/r2d2_500/AP-GeM-LM18_top5/reconstruction', '--output_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/reconstruction', '--Mapper.ba_refine_focal_length', '0', '--Mapper.ba_refine_principal_point', '0', '--Mapper.ba_refine_extra_params', '0', '--Mapper.min_num_matches', '4', '--Mapper.init_min_num_inliers', '4', '--Mapper.abs_pose_min_num_inliers', '4', '--Mapper.abs_pose_min_inlier_ratio', '0.05', '--Mapper.ba_local_max_num_iterations', '50', '--Mapper.abs_pose_max_error', '20', '--Mapper.filter_max_reproj_error', '12']

==============================================================================
Loading database
==============================================================================

Loading cameras... 6 in 0.000s
Loading matches... 30 in 0.000s
Loading images... 16 in 0.000s (connected 12)
Building correspondence graph... in 0.001s (ignored 0)

Elapsed time: 0.000 [minutes]

INFO    ::colmap_localize: Step 4: Export reconstruction results to txt
INFO    ::colmap: ['colmap', 'model_converter', '--input_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/reconstruction', '--output_path', './colmap-localization/r2d2_500/AP-GeM-LM18_top5/AP-GeM-LM18_top5/colmap_localized/reconstruction', '--output_type', 'TXT']
INFO    ::colmap: importing colmap ...
INFO    ::colmap: parsing cameras  ...
16it [00:00, 12230.52it/s]
INFO    ::colmap: saving to kapture  ...
INFO    ::recover_timestamps_and_ids: recover_timestamps_and_ids...
INFO    ::recover_timestamps_and_ids: loading data ...
INFO    ::recover_timestamps_and_ids: recover records and trajectories
INFO    ::recover_timestamps_and_ids: recover sensor ids in sensors
INFO    ::recover_timestamps_and_ids: recover rig ids in rigs
INFO    ::recover_timestamps_and_ids: saving results
INFO    ::recover_timestamps_and_ids: handle image files with a call to transfer_actions
INFO    ::recover_timestamps_and_ids: import image files ...
INFO    ::recover_timestamps_and_ids: done.

Why this difference happen? Does query data's folder structure and name matter when localize pipeline? I think both query data are proper kapture format for localization, and there are no error msgs during process.

yocabon commented 2 years ago

Hi, did you also change the structure of these folders ? ./local_features/r2d2_500/keypoints ./local_features/r2d2_500/descriptors ./global_features/AP-GeM-LM18/global_features

they all use image names (relative paths to records_data) as keys so they must match what's in records_data

Keunmo commented 2 years ago

Oh, I didn't change the structure of them. I see what was the problem. Thank you!

yocabon commented 2 years ago

I should also point out that you should really use -v debug instead of -v info in this kind of scenarios. it would print more logs.

Keunmo commented 2 years ago

Thank you so much. I'll use that.