search

NeoGeographyToolkit / StereoPipeline

The NASA Ames Stereo Pipeline is a suite of automated geodesy & stereogrammetry tools designed for processing planetary imagery captured from orbiting and landed robotic explorers on other planets.
Apache License 2.0
492 stars 173 forks source link

Inprove convergence of convert_pinhole_model with RPC distortion #331

Closed oleg-alexandrov closed 2 years ago

oleg-alexandrov commented 3 years ago

When converting a pinhole model with radial and tangential distortion with a pinhole model with RPC distortion with a command:

convert_pinhole_model input.tif input.tsai --rpc-degree 4 --output-type RPC -o output_rpc.tsai

the obtained model sometimes (not always) has zero RPC coefficients, which shows that it did not converge. Need to figure out why.

A solution can be to first create a low-degree RPC approximation, and then refine it by gradually increasing the degree, instead of trying to right away find a solution for the high degree.

Here's an example of where this fails, with James having provided the distortion. It worked for me when I cooked up the rest of the model. Waiting to see an example of the original model focal length and optical center for which it failed.

VERSION_3 fu = 28.968757624607505 fv = 28.968757624607505 cu = 18.11581494334796 cv = 12.0750534040106 u_direction = 1 0 0 v_direction = 0 1 0 w_direction = 0 0 1 C = 1290593.429554183 -1851555.8545594842 -5945967.2946972586 R = -0.086749194841617583 -0.9739712401063535 -0.20941490071153715 0.94285715281561078 -0.14814812966612889 0.29845019862762323 -0.32170633593196607 -0.17155802261077657 0.93116748133679439 pitch = 0.0063 TSAI k1 = 0.0051253338123196648 k2 = -0.0057740079387240866 p1 = -6.0040372294531705e-05 p2 = 2.7279402066284865e-05

jchollingsworth commented 3 years ago

Hi Oleg,

Here's one of the tsai camera models that is problematic with higher order RPC conversions with convert_pinhole_model: VERSION_4 PINHOLE fu = 4884.2132789899997 fv = 4884.2132789899997 cu = 3695.5039999999999 cv = 3698.4960000000001 u_direction = 1 0 0 v_direction = 0 1 0 w_direction = 0 0 1 C = 4679083.9700235557 1277067.5378684597 4136945.1428247904 R = 0.27197299678208597 -0.62460796930246398 -0.73204888751040653 -0.96218794535798691 -0.18836848465950706 -0.19675281902645492 -0.01500156089036403 0.75788006879118652 -0.65222139990950889 pitch = 1 TSAI k1 = 0.0051253338123196648 k2 = -0.0057740079387240866 p1 = -6.0040372294531705e-05 p2 = 2.7279402066284865e-05

So you'll need to make a fake tif for convert_pinhole_model to work... 7391 by 7397 pixels.

For me, RPC2 conversion works, but RPC3,4,5 fail. convert_pinhole_model 9838.tif 9838.tsai --output-type RPC --rpc-degree 2 -o 9838_RPC2.tsai ... Sample the image by picking one in every 50 pixels. Loading camera model file: ba3-9838.tsai Read a Pinhole camera model. Approximated an TSAI distortion model using a distortion model of type RPC with mean pixel error of 0.869287 and max pixel error of 8.95502. RPC undistortion approximation mean pixel error is 0.00037743 and max pixel error is 0.00293655. Writing output model: test_RPC2.tsai

And for RPC3,4,5: ... Sample the image by picking one in every 50 pixels. Loading camera model file: ba3-9838.tsai Read a Pinhole camera model. Approximated an TSAI distortion model using a distortion model of type RPC with mean pixel error of 2.46587 and max pixel error of 9.95579. RPC undistortion approximation mean pixel error is 0 and max pixel error is 0. Writing output model: test_RPC3.tsai e.g. VERSION_4 PINHOLE fu = 4884.2132789899997 fv = 4884.2132789899997 cu = 3695.5039999999999 cv = 3698.4960000000001 u_direction = 1 0 0 v_direction = 0 1 0 w_direction = 0 0 1 C = 4679083.9700235557 1277067.5378684597 4136945.1428247904 R = 0.27197299678208597 -0.62460796930246398 -0.73204888751040653 -0.96218794535798691 -0.18836848465950706 -0.19675281902645492 -0.01500156089036403 0.75788006879118652 -0.65222139990950889 pitch = 1 RPC rpc_degree = 3 image_size = 7391 7397 distortion_num_x = 0 1 0 0 0 0 0 0 0 0 distortion_den_x = 1 0 0 0 0 0 0 0 0 0 distortion_num_y = 0 0 1 0 0 0 0 0 0 0 distortion_den_y = 1 0 0 0 0 0 0 0 0 0 undistortion_num_x = 0 1 0 0 0 0 0 0 0 0 undistortion_den_x = 1 0 0 0 0 0 0 0 0 0 undistortion_num_y = 0 0 1 0 0 0 0 0 0 0 undistortion_den_y = 1 0 0 0 0 0 0 0 0 0

oleg-alexandrov commented 3 years ago

I could reproduce this. I also added to convert_pinhole_model the option --image-size so it can run with given dimensions without an actual image.

This issue was due to a bug in our Levenberg-Marquardt algorithm implementation, it was giving up too easily when could not converge in several tries.

I also improved the generation of the RPC model and of its inverse by first fitting a model of degree 1, then refining it into a degree 2, etc, up to desired degree. That works better than aiming right away to the highest polynomial degree.

The daily build has the fix.

Here's the kind of output it now produces when trying to fit an RPC model and its inverse:

Compute the RPC distortion model starting at degree 1 and then refine it until reaching degree 5. Approximated an TSAI distortion model using a distortion model of type RPC of degree 1 with mean, max, and norm of pixel error of 0.924945, 13.0868, and 219.415. Approximated an TSAI distortion model using a distortion model of type RPC of degree 2 with mean, max, and norm of pixel error of 0.870284, 8.9979, and 163.512. Approximated an TSAI distortion model using a distortion model of type RPC of degree 3 with mean, max, and norm of pixel error of 0.870284, 8.99677, and 163.341. Approximated an TSAI distortion model using a distortion model of type RPC of degree 4 with mean, max, and norm of pixel error of 0.87263, 9.09802, and 162.958. Approximated an TSAI distortion model using a distortion model of type RPC of degree 5 with mean, max, and norm of pixel error of 0.87263, 9.09802, and 162.957.

Will compute undistortion for degree 1 and then refine it. RPC undistortion approximation of degree 1 has mean, max, and norm of pixel error of 0.780414, 4.18721, and 134.963. RPC undistortion approximation of degree 2 has mean, max, and norm of pixel error of 0.020725, 0.148453, and 3.97439. RPC undistortion approximation of degree 3 has mean, max, and norm of pixel error of 0.0207259, 0.148622, and 3.9742. RPC undistortion approximation of degree 4 has mean, max, and norm of pixel error of 0.0207259, 0.148622, and 3.9742. RPC undistortion approximation of degree 5 has mean, max, and norm of pixel error of 0.0207259, 0.148622, and 3.9742.

jchollingsworth commented 3 years ago

Wow, that's incredible. I'll test and report back if I find anything weird.

Many thanks!