3D reconstruction with own data

[slz929]

I want to construct my own sequnce data captured by cameras mounted on a vehicle. I use the script in pipelines/reconstruction.py

steps:

1. I prepared 2500 images with size of 1280 * 720, extract NetVLAD vectors, get 20 paris from retrieval for every image.

   global_descriptors = extract_features.main(retrieval_conf, dataset, outputs)
   pairs_from_retrieval.main(
   global_descriptors, sfm_pairs, 20,
   query_list= images, db_list= images, db_descriptors= global_descriptors)

2. I extract the Superpoint and match by NN.

   'superpoint_zgc': {
       'output': 'feats-superpoint-n4096-r1024',
       'model': {
           'name': 'superpoint',
           'nms_radius': 3,
           'max_keypoints': 4096,
       },
       'preprocessing': {
           'grayscale': True,
           'resize_max': 1280,
       },
   },

   'NN-superpoint': {
       'output': 'matches-NN-mutual-dist.7',
       'model': {
           'name': 'nearest_neighbor',
           'do_mutual_check': True,
           'distance_threshold': 0.7,
   },

3. Then I construct with single camera.

   reconstruction.main(sfm_dir, dataset, sfm_pairs, feature_path, match_path, 
       colmap_path= 'colmap/build/src/exe/colmap', single_camera= True)

But I get the output, it seems that very few images were used or stored in bin file. And I get 7 sub-models with small size.

Elapsed time: 904.367 [minutes]
[12/29/2021 08:55:22 INFO] Reconstructed 7 models.
[12/29/2021 08:55:22 INFO] Largest model is #0 with 1 images.
[12/29/2021 08:55:22 INFO] Statistics:
{'mean_reproj_error': 1.312922,
 'mean_track_length': 6.724747,
 'num_input_images': 2569,
 'num_observations': 34619,
 'num_observations_per_image': 2884.916667,
 'num_reg_images': 12,
 'num_sparse_points': 5148}

Compared to the output of Aachen dataset.

[11/24/2021 16:03:11 INFO] Statistics:
{'mean_reproj_error': 1.322519,
 'mean_track_length': 5.709355,
 'num_observations': 5203032,
 'num_observations_per_image': 1202.179298,
 'num_reg_images': 4328,
 'num_sparse_points': 911317}

And my result with my own data cause the wrong localization and query image retrieval. Looking forward to your reply! Thanks sincerely.

[sarlinpe]

If you have sequence data, you should change the way the image pairs are selected and use both:

• sequential pairs between each image and its K previous ones. We don't have a script for that, you're welcome to PR one with a new script pairs_from_sequence.
• loop closure pairs with image retrieval, triggered every N frame (depending on the framerate + motion magnitude).

Overall, you want to make sure that there are more constraints from the sequence and few sparse loop closures.