panorama test without theory

[zebrajack]

since narrow perspective cannot ensure the triplet. we choose 3 panoramas. but I know the theory cannot support the vps in panoramas. but it genrated some results.

Though the result looks like it is not that bad. so does it mean some conincident?

All the dataset can be download below. https://drive.google.com/open?id=14eerqQs2g0LudnXJsMCDBsMoo0g4sQOl

[ySalaun]

Hi,

Though I don't really know what you expect from me, I can tell you that the vanishing points are wrongly estimated on your pictures (vertical one is correct but horizontal is a mixed of multiple horizontal directions) However, vanishing points are only used to limit the number of line pairs for calibration estimation. It means that the soft should work (even if slower) with every segment associated to the same VP (even if it's false).

Best,

Yohann

[zebrajack]

yeah. just want to ensure that the VP are wrong and the result is not right.

[haopo2005]

Hi, @ySalaun We'd like you to help us verify the correctness of global pose from the RECONSTRUCTION application based on our image set (both panorama and perspective image we've tried) There are not enough feature points in our images and we'd like to use the line features to compute camera pose. And your paper said , it can handle the situation of less feature points. And the CALIBRATION application has executed as we expected. As you can see, the lines are the final kept feature. jst@jst-pc:~/share/project/LineSfM-master/build$ ./CALIBRATION -d ../test_image/ -i ../images2.txt -v 1 4 1 /home/jst/share/project/LineSfM-master/test_image/1.PNG 2 /home/jst/share/project/LineSfM-master/test_image/2.PNG 3 /home/jst/share/project/LineSfM-master/test_image/3.PNG 4 /home/jst/share/project/LineSfM-master/test_image/4.PNG reading intrinsic parameters reading global poses computing relative poses LOAD DETECTIONS AND MATCHES picture 0: 1

• read points
• read segments
• read vanishing points picture 1: 2
• read points
• read segments
• read vanishing points picture 2: 3
• read points
• read segments
• read vanishing points picture 3: 4
• read points
• read segments
• read vanishing points pictures 1 and 2
• read point matches
• read line matches pictures 2 and 3
• read point matches
• read line matches pictures 3 and 4
• read point matches
• read line matches COMPUTE POSES pictures 1 and 2 === BETTER MODEL FOUND === NFA: -33.3503

  inliers: 82

• points: 15/20
• lines: 19/30
• parallel pairs: 67/143 === BETTER MODEL FOUND === NFA: -43.6666

  inliers: 18

• points: 18/20
• lines: 28/30
• parallel pairs: 0/143 === BETTER MODEL FOUND === NFA: -46.1029

  inliers: 26

• points: 20/20
• lines: 28/30
• parallel pairs: 6/143 === BETTER MODEL FOUND === NFA: -53.4207

  inliers: 49

• points: 20/20
• lines: 23/30
• parallel pairs: 29/143 === BETTER MODEL FOUND === NFA: -58.7462

  inliers: 46

• points: 19/20
• lines: 20/30
• parallel pairs: 27/143 final kept hypothesis: LINES WARNING: Logging before InitGoogleLogging() is written to STDERR E0919 10:04:00.823555 4555 trust_region_minimizer.cc:91] Terminating: Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5

Solver Summary (v 1.13.0-eigen-(3.3.4)-lapack-suitesparse-(4.4.6)-cxsparse-(3.1.4)-eigensparse-openmp-no_tbb)

                                 Original                  Reduced

Parameter blocks 2 2 Parameters 6 6 Residual blocks 46 46 Residual 46 46

Minimizer TRUST_REGION

Dense linear algebra library EIGEN Trust region strategy LEVENBERG_MARQUARDT

                                    Given                     Used

Linear solver DENSE_QR DENSE_QR Threads 1 1 Linear solver threads 1 1 Linear solver ordering AUTOMATIC 2

Cost: Initial 8.625961e-03

Minimizer iterations 5 Successful steps 1 Unsuccessful steps 4

Time (in seconds): Preprocessor 0.011321

Residual evaluation 0.000000 Jacobian evaluation 0.007050 Linear solver 0.009047 Minimizer 0.016271

Postprocessor 0.000009 Total 0.027602

Termination: FAILURE (Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5)

pictures 2 and 3 === BETTER MODEL FOUND === NFA: -69.6865

inliers: 128

• points: 32/35
• lines: 30/42
• parallel pairs: 96/214 === BETTER MODEL FOUND === NFA: -81.3461

  inliers: 26

• points: 26/35
• lines: 35/42
• parallel pairs: 0/214 === BETTER MODEL FOUND === NFA: -82.0272

  inliers: 37

• points: 22/35
• lines: 27/42
• parallel pairs: 15/214 === BETTER MODEL FOUND === NFA: -90.9075

  inliers: 30

• points: 30/35
• lines: 34/42
• parallel pairs: 0/214 === BETTER MODEL FOUND === NFA: -102.702

  inliers: 99

• points: 29/35
• lines: 26/42
• parallel pairs: 70/214 === BETTER MODEL FOUND === NFA: -105.49

  inliers: 31

• points: 31/35
• lines: 29/42
• parallel pairs: 0/214 === BETTER MODEL FOUND === NFA: -112.041

  inliers: 34

• points: 30/35
• lines: 27/42
• parallel pairs: 4/214 === BETTER MODEL FOUND === NFA: -126.755

  inliers: 88

• points: 31/35
• lines: 26/42
• parallel pairs: 57/214 final kept hypothesis: LINES E0919 10:04:01.145869 4555 trust_region_minimizer.cc:91] Terminating: Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5

Solver Summary (v 1.13.0-eigen-(3.3.4)-lapack-suitesparse-(4.4.6)-cxsparse-(3.1.4)-eigensparse-openmp-no_tbb)

                                 Original                  Reduced

Parameter blocks 2 2 Parameters 6 6 Residual blocks 88 88 Residual 88 88

Minimizer TRUST_REGION

Dense linear algebra library EIGEN Trust region strategy LEVENBERG_MARQUARDT

                                    Given                     Used

Linear solver DENSE_QR DENSE_QR Threads 1 1 Linear solver threads 1 1 Linear solver ordering AUTOMATIC 2

Cost: Initial 5.571573e-03

Minimizer iterations 5 Successful steps 1 Unsuccessful steps 4

Time (in seconds): Preprocessor 0.000013

Residual evaluation 0.000000 Jacobian evaluation 0.000052 Linear solver 0.000032 Minimizer 0.000112

Postprocessor 0.000002 Total 0.000127

Termination: FAILURE (Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5)

pictures 3 and 4 === BETTER MODEL FOUND === NFA: 8.59071

inliers: 18

• points: 15/21
• lines: 25/30
• parallel pairs: 3/116 === BETTER MODEL FOUND === NFA: -22.7967

  inliers: 22

• points: 20/21
• lines: 8/30
• parallel pairs: 2/116 === BETTER MODEL FOUND === NFA: -34.9052

  inliers: 20

• points: 20/21
• lines: 22/30
• parallel pairs: 0/116 === BETTER MODEL FOUND === NFA: -41.2829

  inliers: 21

• points: 21/21
• lines: 21/30
• parallel pairs: 0/116 === BETTER MODEL FOUND === NFA: -47.1451

  inliers: 21

• points: 21/21
• lines: 23/30
• parallel pairs: 0/116 === BETTER MODEL FOUND === NFA: -47.5883

  inliers: 21

• points: 21/21
• lines: 22/30
• parallel pairs: 0/116 === BETTER MODEL FOUND === NFA: -48.1566

  inliers: 21

• points: 21/21
• lines: 23/30
• parallel pairs: 0/116 === BETTER MODEL FOUND === NFA: -48.6653

  inliers: 21

• points: 21/21
• lines: 23/30
• parallel pairs: 0/116 final kept hypothesis: LINES E0919 10:04:01.410118 4555 trust_region_minimizer.cc:91] Terminating: Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5

Solver Summary (v 1.13.0-eigen-(3.3.4)-lapack-suitesparse-(4.4.6)-cxsparse-(3.1.4)-eigensparse-openmp-no_tbb)

                                 Original                  Reduced

Parameter blocks 2 2 Parameters 6 6 Residual blocks 21 21 Residual 21 21

Minimizer TRUST_REGION

Dense linear algebra library EIGEN Trust region strategy LEVENBERG_MARQUARDT

                                    Given                     Used

Linear solver DENSE_QR DENSE_QR Threads 1 1 Linear solver threads 1 1 Linear solver ordering AUTOMATIC 2

Cost: Initial 1.272141e-03

Minimizer iterations 5 Successful steps 1 Unsuccessful steps 4

Time (in seconds): Preprocessor 0.000010

Residual evaluation 0.000000 Jacobian evaluation 0.000031 Linear solver 0.000017 Minimizer 0.000077

Postprocessor 0.000002 Total 0.000089

Termination: FAILURE (Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5)

PROCESSED IN 0.834883 images 0 and 1 Error in rotation : 120 Error in translation : nan images 1 and 2 Error in rotation : 120 Error in translation : nan images 2 and 3 Error in rotation : 120 Error in translation : nan AVERAGE RESULTS: Rotation: 120 Translation: nan jst@jst-pc:~/share/project/LineSfM-master/build$

As followed pic, the global poses of your TEST_IMAGE (4 outdoor images with rich features) are right

And this ours is not correct,

The global poses are extracted before bundle adjustment in the RECONSTRUCTION application. (we'd like the first image's coord to be 0,0,0.)

[haopo2005]

Here is a small batch of our images.

[haopo2005]

Besides we reuse the calibration matrix K from your TEST_IMAGE to compute everything, dose it matter?

[ySalaun]

Hi,

This seems to be a tough scene so you should check:

• point matches: even if 2 only matches are required, they need to be correct
• line matches: the matching does not need to be perfect but segments that are matched together need to belong to the same vanishing point
• vanishing points: you should check in the resulting images if the clusters are correct enough

About the K matrix, the result won't be correct if you use the wrong matrix. However, I'm not sure it will solve your problem in this case. To compute your own K matrix you should follow the formula below: K = | focal_length (mm) picture_height (pixel nb) / sensor_height(mm), 0, 0.5picture_height (pixel nb) | | 0, focal_length (mm) picture_width (pixel nb) / sensor_width(mm), 0.5picture_width (pixel nb) | | 0,0,1 | (Note that the parameters can be found on your exif (for focal length) and internet (for sensor information))

Best,

Yohann