getExt1 unable to evaluate residual block with Ceres Solver

[VisionaryMind]

We have executed your process faithfully, generating eight RGB images with intrinsic / distortion data, picking points on both the images and on the relevant PCD poindclouds. Everything is in order, but the extrinsics cannot be extracted. We are using Ceres 2.0.0 and PCL 1.8.1. Your instructions do not indicate which versions you require.

The following is the console log output:

Terminal Log - Extrinsics Capture

`started roslaunch server http://ubuntu:40985/ SUMMARY ======== PARAMETERS * /error_threshold: 12 * /extrinsic_path: /home/visionarymi... * /init_value: [0.0, -1.0, 0.0, ... * /input_lidar_path: /home/visionarymi... * /input_photo_path: /home/visionarymi... * /intrinsic_path: /home/visionarymi... * /rosdistro: melodic * /rosversion: 1.14.10 NODES / getExt1 (camera_lidar_calibration/getExt1) auto-starting new master process[master]: started with pid [21409] ROS_MASTER_URI=http://localhost:11311 setting /run_id to 40fcc524-5b86-11eb-9025-000c29a7eb2a process[rosout-1]: started with pid [21420] started core service [/rosout] process[getExt1-2]: started with pid [21423] Get the parameters from the launch file 3.604 0.348 0.127 868 453 3.632 -0.047 0.12 968 450 3.579 0.355 -0.44 866.199 602.522 3.591 -0.04 -0.439 970.089 603.662 3.221 0.954 0.16 680.35 436.59 3.332 0.584 0.149 797.771 440.533 3.205 0.967 -0.413 678.422 607.985 3.309 0.589 -0.424 796.386 602.285 3.535 0.05 0.133 943 449 3.449 -0.324 0.126 1045.36 444.462 3.512 0.057 -0.434 941.998 603.281 3.472 -0.325 -0.441 1046.23 607.426 4.471 0.577 0.089 837.117 471.398 4.61 0.191 0.089 918 472 4.448 0.58 -0.464 841 593 4.483 0.202 -0.471 921 590 4.332 1.24 0.113 697 466 4.449 0.87 0.097 778.264 469.58 4.245 1.215 -0.441 693.186 595.975 0 0 0 778.178 594.362 4.332 1.24 0.113 837.424 429.42 4.449 0.87 0.097 960.897 429.78 4.245 1.215 -0.441 840.903 614.713 0 0 0 962 608 2.864 0.723 0.154 724.223 423.595 3.021 0.371 0.159 850.246 430.882 2.861 0.724 -0.418 725.17 616.304 3.011 0.373 -0.413 850.281 615.028 2.697 0.19 0.166 893.511 414.386 2.648 -0.194 0.164 1032.19 409.463 2.676 0.189 -0.405 894.237 616.112 2.614 -0.198 -0.408 1035.77 618.983 2.203 0.201 0.184 867.476 381.646 2.215 -0.175 0.184 1035.6 380.578 2.174 0.208 -0.38 870.21 632.406 2.202 -0.18 -0.384 1039.56 630.145 2.094 0.333 0.192 799 373 2.264 -0.023 0.186 969.316 384.139 2.075 0.335 -0.38 807.088 635.786 2.239 -0.021 -0.388 972.289 628.854 WARNING: Logging before InitGoogleLogging() is written to STDERR W0120 17:16:14.738849 21423 residual_block.cc:129] Error in evaluating the ResidualBlock. There are two possible reasons. Either the CostFunction did not evaluate and fill all residual and jacobians that were requested or there was a non-finite value (nan/infinite) generated during the or jacobian computation. Residual Block size: 2 parameter blocks x 2 residuals For each parameter block, the value of the parameters are printed in the first column and the value of the jacobian under the corresponding residual. If a ParameterBlock was held constant then the corresponding jacobian is printed as 'Not Computed'. If an entry of the Jacobian/residual array was requested but was not written to by user code, it is indicated by 'Uninitialized'. This is an error. Residuals or Jacobian values evaluating to Inf or NaN is also an error. Residuals: -nan -nan Parameter Block 0, size: 4 0.5 | -nan -nan -0.5 | -nan -nan 0.5 | -nan -nan 0.5 | -nan -nan Parameter Block 1, size: 3 0 | -nan -nan 0 | -nan -nan 0 | -nan -nan E0120 17:16:14.738970 21423 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed. Ceres Solver Report: Iterations: 0, Initial cost: -1.000000e+00, Final cost: -1.000000e+00, Termination: FAILURE 0 -1 0 0 0 -1 1 0 0 0 0 0 Use the extrinsic result to reproject the data Data 0 has a error bigger than the threshold xyz are 3.604 0.348 0.127 ErrorU is 8.04004 errorV is 61.6647 ********************** Data 1 has a error bigger than the threshold xyz are 3.632 -0.047 0.12 ErrorU is 8.23987 errorV is 66.6819 ********************** Data 2 has a error bigger than the threshold xyz are 3.579 0.355 -0.44 ErrorU is 7.43412 errorV is 57.2514 ********************** Data 3 has a error bigger than the threshold xyz are 3.591 -0.04 -0.439 ErrorU is 4.50225 errorV is 55.4791 ********************** Data 4 has a error bigger than the threshold xyz are 3.221 0.954 0.16 ErrorU is 13.0226 errorV is 64.832 ********************** Data 5 has a error bigger than the threshold xyz are 3.332 0.584 0.149 ErrorU is 6.2434 errorV is 65.4356 ********************** Data 6 has a error bigger than the threshold xyz are 3.205 0.967 -0.413 ErrorU is 9.88592 errorV is 57.221 ********************** Data 7 has a error bigger than the threshold xyz are 3.309 0.589 -0.424 ErrorU is 5.13097 errorV is 62.2552 ********************** Data 8 has a error bigger than the threshold xyz are 3.535 0.05 0.133 ErrorU is 8.45544 errorV is 63.4766 ********************** Data 9 has a error bigger than the threshold xyz are 3.449 -0.324 0.126 ErrorU is 4.99999 errorV is 69.0158 ********************** Data 10 has a error bigger than the threshold xyz are 3.512 0.057 -0.434 ErrorU is 7.54875 errorV is 57.0768 ********************** Data 11 has a error bigger than the threshold xyz are 3.472 -0.325 -0.441 ErrorU is 3.8242 errorV is 56.0874 ********************** Data 12 has a error bigger than the threshold xyz are 4.471 0.577 0.089 ErrorU is 9.18836 errorV is 57.3323 ********************** Data 13 has a error bigger than the threshold xyz are 4.61 0.191 0.089 ErrorU is 8.4856 errorV is 57.281 ********************** Data 14 has a error bigger than the threshold xyz are 4.448 0.58 -0.464 ErrorU is 4.07758 errorV is 49.6893 ********************** Data 15 has a error bigger than the threshold xyz are 4.483 0.202 -0.471 ErrorU is 2.16626 errorV is 53.3746 ********************** Data 16 has a error bigger than the threshold xyz are 4.332 1.24 0.113 ErrorU is 5.46783 errorV is 57.0621 ********************** Data 17 has a error bigger than the threshold xyz are 4.449 0.87 0.097 ErrorU is 7.1934 errorV is 57.4105 ********************** Data 18 has a error bigger than the threshold xyz are 4.245 1.215 -0.441 ErrorU is 9.30271 errorV is 46.3202 ********************** Data 20 has a error bigger than the threshold xyz are 4.332 1.24 0.113 ErrorU is 134.956 errorV is 93.6421 ********************** Data 21 has a error bigger than the threshold xyz are 4.449 0.87 0.097 ErrorU is 175.44 errorV is 97.2105 ********************** Data 22 has a error bigger than the threshold xyz are 4.245 1.215 -0.441 ErrorU is 138.414 errorV is 27.5822 ********************** Data 24 has a error bigger than the threshold xyz are 2.864 0.723 0.154 ErrorU is 9.17202 errorV is 74.0685 ********************** Data 25 has a error bigger than the threshold xyz are 3.021 0.371 0.159 ErrorU is 1.77573 errorV is 67.8268 ********************** Data 26 has a error bigger than the threshold xyz are 2.861 0.724 -0.418 ErrorU is 7.66301 errorV is 64.7189 ********************** Data 27 has a error bigger than the threshold xyz are 3.011 0.373 -0.413 ErrorU is 0.759649 errorV is 57.7946 ********************** Data 28 has a error bigger than the threshold xyz are 2.697 0.19 0.166 ErrorU is 6.42211 errorV is 76.1413 ********************** Data 29 has a error bigger than the threshold xyz are 2.648 -0.194 0.164 ErrorU is 0.751157 errorV is 80.7124 ********************** Data 30 has a error bigger than the threshold xyz are 2.676 0.189 -0.405 ErrorU is 5.53217 errorV is 69.7203 ********************** Data 31 has a error bigger than the threshold xyz are 2.614 -0.198 -0.408 ErrorU is 2.05906 errorV is 71.1952 ********************** Data 32 has a error bigger than the threshold xyz are 2.203 0.201 0.184 ErrorU is 13.4323 errorV is 88.7242 ********************** Data 33 has a error bigger than the threshold xyz are 2.215 -0.175 0.184 ErrorU is 1.09482 errorV is 90.2074 ********************** Data 34 has a error bigger than the threshold xyz are 2.174 0.208 -0.38 ErrorU is 6.63821 errorV is 74.9367 ********************** Data 35 has a error bigger than the threshold xyz are 2.202 -0.18 -0.384 ErrorU is 0.360415 errorV is 76.8253 ********************** Data 36 has a error bigger than the threshold xyz are 2.094 0.333 0.192 ErrorU is 19.879 errorV is 89.877 ********************** Data 37 has a error bigger than the threshold xyz are 2.264 -0.023 0.186 ErrorU is 4.37864 errorV is 87.4853 ********************** Data 38 has a error bigger than the threshold xyz are 2.075 0.335 -0.38 ErrorU is 9.57661 errorV is 79.2072 ********************** Data 39 has a error bigger than the threshold xyz are 2.239 -0.021 -0.388 ErrorU is 0.691957 errorV is 77.1114 ********************** u average error is: nan v average error is: nan The reprojection error bigger than the threshold, extrinsic result seems not ok [getExt1-2] process has finished cleanly log file: /home/visionarymind/.ros/log/40fcc524-5b86-11eb-9025-000c29a7eb2a/getExt1-2*.log `

We see this same problem if we use your provided sample data, so the issue is likely not in the data itself.

[VisionaryMind]

This issue occurs when one of the picked PCD points is 0,0,0. This can happen if a space in between points is clicked and then transferred to the points file.

[Shibaditya99]

Hi @VisionaryMind , I don't have any 0,0,0 PCD points, still getting the same error. Do you know from where I can get those initial extrinsic values of LiDAR?