search

openMVG / openMVG

open Multiple View Geometry library. Basis for 3D computer vision and Structure from Motion.
Mozilla Public License 2.0
5.56k stars 1.65k forks source link

[Spherical SfM] Relative translation failure at openMVG_main_SfM #2032

Closed DiantaoTu closed 2 years ago

DiantaoTu commented 2 years ago

I got an error doing global SfM on both test data (360_Flat and 360_school). I used the latest version of openMVG with Ubuntu 20.04 and Eigen 3.4.0. Here are the commands I used on the 360_Flat dataset.

  1. ImageListing 
    sfm_bin=/home/tdt/library/openMVG/build/Linux-x86_64-Release
$sfm_bin/openMVG_main_SfMInit_ImageListing -i ./images -o ./matches -f 1 -c 7

    The output is

    
INFO: [main_SfMInit_ImageListing.cpp:197]  You called : /home/tdt/library/openMVG/build/Linux-x86_64-Release/openMVG_main_SfMInit_ImageListing
--imageDirectory ./images
--sensorWidthDatabase 
--outputDirectory ./matches
--focal 1
--intrinsics 
--camera_model 7
--group_camera_model 1
--use_pose_prior 0
--prior_weights 1.0;1.0;1.0
--gps_to_xyz_method 0
INFO: [loggerprogress.hpp:79] [- Listing images -] 50%
INFO: [loggerprogress.hpp:79] [- Listing images -] 100%
WARNING: [main_SfMInit_ImageListing.cpp:458] Warning & Error messages:
README.md: Unkown image file format.

INFO: [main_SfMInit_ImageListing.cpp:478] SfMInit_ImageListing report: listed #File(s): 12 usable #File(s) listed in sfm_data: 11 usable #Intrinsic(s) listed in sfm_data: 1

2. ComputeFeatures
`$sfm_bin/openMVG_main_ComputeFeatures -i ./matches/sfm_data.json -o ./matches`
The output is

INFO: [main_ComputeFeatures.cpp:121] You called : /home/tdt/library/openMVG/build/Linux-x86_64-Release/openMVG_main_ComputeFeatures --input_file ./matches/sfm_data.json --outdir ./matches --describerMethod SIFT --upright 0 --describerPreset NORMAL --force 0 --numThreads 0

INFO: [loggerprogress.hpp:79] [- EXTRACT FEATURES -] 100% INFO: [main_ComputeFeatures.cpp:344] Task done in (s): 27

3. PairGenerator
`$sfm_bin/openMVG_main_PairGenerator -i ./matches/sfm_data.json -o ./matches/pairs.bin`
The output is

You called: /home/tdt/library/openMVG/build/Linux-x86_64-Release/openMVG_main_PairGenerator --input_file : ./matches/sfm_data.json --output_file : ./matches/pairs.bin Optional parameters --pair_mode : EXHAUSTIVE --contiguous_count : -1

Loading scene.Computing pairs. Saving pairs.

4. ComputeMatches
`$sfm_bin/openMVG_main_ComputeMatches -i ./matches/sfm_data.json -p ./matches/pairs.bin  -o ./matches/matches.putative.bin`
The output is

INFO: [main_ComputeMatches.cpp:114] You called : /home/tdt/library/openMVG/build/Linux-x86_64-Release/openMVG_main_ComputeMatches --input_file ./matches/sfm_data.json --output_file ./matches/matches.putative.bin --pair_list ./matches/pairs.bin Optional parameters: --force 0 --ratio 0.8 --nearest_matching_method AUTO --cache_size unlimited --preemptive_feature_used/count 0 / 200 INFO: [loggerprogress.hpp:79] [- Regions Loading -] 100% INFO: [main_ComputeMatches.cpp:217] - PUTATIVE MATCHES - INFO: [main_ComputeMatches.cpp:238] Using FAST_CASCADE_HASHING_L2 matcher INFO: [main_ComputeMatches.cpp:317] Running matching on #pairs: 55 INFO: [Cascade_Hashing_Matcher_Regions.cpp:238] Using the OPENMP thread interface INFO: [loggerprogress.hpp:79] [- Matching -] 20% INFO: [loggerprogress.hpp:79] [- Matching -] 40% INFO: [loggerprogress.hpp:79] [- Matching -] 60% INFO: [loggerprogress.hpp:79] [- Matching -] 80% INFO: [loggerprogress.hpp:79] [- Matching -] 100% INFO: [main_ComputeMatches.cpp:363] Task (Regions Matching) done in (s): 0 INFO: [main_ComputeMatches.cpp:366] #Putative pairs: 55 INFO: [graph_stats.hpp:53] Graph statistics:

nodes: 11

#cc: 1
#singleton: 0
Node degree statistics: min: 10, max: 10, mean: 10, median: 10
5. GeometricFilter
`$sfm_bin/openMVG_main_GeometricFilter -i ./matches/sfm_data.json -m ./matches/matches.putative.bin -g a -o ./matches/matches.a.bin
`
The output is

INFO: [main_GeometricFilter.cpp:132] You called : /home/tdt/library/openMVG/build/Linux-x86_64-Release/openMVG_main_GeometricFilter --input_file: ./matches/sfm_data.json --matches: ./matches/matches.putative.bin --output_file: ./matches/matches.a.bin Optional parameters: --input_pairs
--output_pairs
--force false --geometric_model a --guided_matching 0 --cache_size unlimited INFO: [loggerprogress.hpp:79] [- Regions Loading -] 100% INFO: [loggerprogress.hpp:79] [- Geometric filtering -] 20% INFO: [loggerprogress.hpp:79] [- Geometric filtering -] 40% INFO: [loggerprogress.hpp:79] [- Geometric filtering -] 60% INFO: [loggerprogress.hpp:79] [- Geometric filtering -] 80% INFO: [loggerprogress.hpp:79] [- Geometric filtering -] 100% INFO: [graph_stats.hpp:53] Graph statistics:

nodes: 11

#cc: 1
#singleton: 0
Node degree statistics: min: 8, max: 10, mean: 9, median: 10

INFO: [main_GeometricFilter.cpp:384] Task done in (s): 0 INFO: [main_GeometricFilter.cpp:387] Export Adjacency Matrix of the pairwise's geometric matches

6. Global SfM

mkdir reconstruction $sfm_bin/openMVG_main_SfM --sfm_engine GLOBAL --input_file ./matches/sfm_data.json --match_file ./matches/matches.a.bin --output_dir ./reconstruction

The output is

INFO: [main_SfM.cpp:135]

Structure from Motion:

INFO: [loggerprogress.hpp:79] [- Features Loading -] 100% INFO: [connectedComponent.hpp:99] CleanGraph_KeepLargestBiEdge_Nodes():: => connected Component: 1 INFO: [connectedComponent.hpp:118] Connected component of size: 11 INFO: [loggerprogress.hpp:79] [- Relative pose computation -] 10% WARNING: Logging before InitGoogleLogging() is written to STDERR W0320 11:06:48.018270 739117 levenberg_marquardt_strategy.cc:115] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. Lots of the same errors ... W0320 11:06:48.449185 739118 levenberg_marquardt_strategy.cc:115] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. INFO: [relative_pose_engine.cpp:224] Relative motion computation took: 487(ms) INFO: [sfm_global_engine_relative_motions.cpp:195]

Global rotations computation:

relative rotations: 50

global rotations: 11

INFO: [GlobalSfM_rotation_averaging.cpp:245] Statistics about rotation triplets: min: 0.0198883 mean: 2.30856 median: 0.690624 max: 25.3382 0 | 90 1.27 | 9 2.53 | 9 3.8 | 4 5.07 | 5 6.33 | 1 7.6 | 0 8.87 | 0 10.1 | 0 11.4 | 0 12.7 | 0 13.9 | 0 15.2 | 0 16.5 | 0 17.7 | 0 19 | 0 20.3 | 1 21.5 | 0 22.8 | 3 24.1 | 1 25.3

Triplets filtering based on unit cycle rotation composition error:

Triplets before: 124

Triplets after: 112

INFO: [GlobalSfM_rotation_averaging.cpp:251]

Edges removed by triplet inference: 2

INFO: [connectedComponent.hpp:99] CleanGraph_KeepLargestBiEdge_Nodes():: => connected Component: 1 INFO: [connectedComponent.hpp:118] Connected component of size: 11 INFO: [sfm_global_engine_relative_motions.cpp:212] Found #global_rotations: 11 Timing: 0 seconds INFO: [sfm_global_engine_relative_motions.cpp:241] Relative/Global degree rotations residual errors {0,23.5277}: 0 | 44 1.18 | 2 2.35 | 1 3.53 | 1 4.71 | 0 5.88 | 1 7.06 | 0 8.23 | 0 9.41 | 0 10.6 | 0 11.8 | 0 12.9 | 0 14.1 | 0 15.3 | 0 16.5 | 0 17.6 | 0 18.8 | 0 20 | 0 21.2 | 0 22.4 | 0 23.5

INFO: [sfm_global_engine_relative_motions.cpp:247] Relative/Global degree rotations residual errors {0,5}: 0 | 18 0.25 | 16 0.5 | 5 0.75 | 4 1 | 2 1.25 | 1 1.5 | 0 1.75 | 0 2 | 0 2.25 | 0 2.5 | 0 2.75 | 0 3 | 1 3.25 | 0 3.5 | 1 3.75 | 0 4 | 0 4.25 | 0 4.5 | 0 4.75 | 0 5

INFO: [sfm_global_engine_relative_motions.cpp:254] Statistics about global rotation evaluation: min: 0.0231026 mean: 1.08078 median: 0.326402 max: 23.5277

INFO: [GlobalSfM_translation_averaging.cpp:305]

Relative translations computation:

INFO: [GlobalSfM_translation_averaging.cpp:364] #Triplets: 139 INFO: [loggerprogress.hpp:79] [Relative translations computation (edge coverage algorithm)] 10% INFO: [loggerprogress.hpp:79] [Relative translations computation (edge coverage algorithm)] 40% INFO: [loggerprogress.hpp:79] [Relative translations computation (edge coverage algorithm)] 50% INFO: [loggerprogress.hpp:79] [Relative translations computation (edge coverage algorithm)] 60% INFO: [loggerprogress.hpp:79] [Relative translations computation (edge coverage algorithm)] 90% INFO: [loggerprogress.hpp:79] [Relative translations computation (edge coverage algorithm)] 100% INFO: [GlobalSfM_translation_averaging.cpp:581] TRIPLET COVERAGE TIMING:

-- #Relative triplet of translations estimates: 0 computed from 139 triplets. -- resulting in 0 translations estimation. -- time to compute triplets of relative translations: 0 seconds.

INFO: [connectedComponent.hpp:99] CleanGraph_KeepLargestBiEdge_Nodes():: => connected Component: 0 INFO: [GlobalSfM_translation_averaging.cpp:107]

Global translations computation:

It seems that the error occurred here #Relative triplet of translations estimates: 0 computed from 139 triplets. in relative translation estimation. I wonder how this problem should be solved.

Thanks!

pmoulon commented 2 years ago

Hello @DiantaoTu , the Global SfM pipeline is working only with Pinhole camera due to the triplet translation estimation step. We could try to update it but I have no time to do it yet.

Please continue to use the Sequential pipeline for now on with Spherical images, or else, skip the triplet estimation and use bi-edge translation if your graph is bi-edge connected.

DiantaoTu commented 2 years ago

Thanks a lot!

pmoulon commented 2 years ago

Please share some reconstruction screenshot if you can, we always appreciate see the reconstruction our user can do ;-)