hku-mars / mlcc

Fast and Accurate Extrinsic Calibration for Multiple LiDARs and Cameras
GNU General Public License v2.0
495 stars 112 forks source link

Optimization crash during `extrinsic_refine` step #21

Closed jun-zhang-cyngn closed 1 year ago

jun-zhang-cyngn commented 1 year ago

Hi there,

I am following the calibration steps with the provided dataset ('scene2') and ran into some branch in the extrinsic_refine step . Could you please share some thoughts on the issue? I have tried the scene1 and got the same issue.

Thanks

roslaunch mlcc extrinsic_refine.launch

process[extrinsic_refine-1]: started with pid [945409]
process[rviz-2]: started with pid [945410]
---------------------
iteration 0
extrinsic_refine: /home/cyngn/github/catkin_mlcc_ws/src/mlcc/include/extrinsic_refine.hpp:178: void EXTRIN_OPTIMIZER::optimize(): Assertion `!std::isnan(residual2)' failed.
jun-zhang-cyngn commented 1 year ago

The problem comes from here It looks like we have an underdetermined system.

cv::solve(matA, matB, matX, cv::DECOMP_QR);
matA: 
[4668.396019965554, 762.2615729792876, 443.5128503180619, 25.7672809111645, -50.61608196270411, -170.2821341374004, 0, 0, 0, 0, 0, 0;
 762.2615729792853, 5517.516799518293, 331.9476116036997, 13.71723803017447, 36.60771030027582, -285.2359537322813, 0, 0, 0, 0, 0, 0;
 443.5128503180659, 331.9476116037035, 1141.239173941337, 57.1814471478689, 14.49662191019416, -18.02984394798824, 0, 0, 0, 0, 0, 0;
 25.76728091116443, 13.71723803017436, 57.18144714786875, 4.913689564802027, 1.673394500703981, 0.1322514847335978, 0, 0, 0, 0, 0, 0;
 -50.61608196270406, 36.60771030027578, 14.49662191019422, 1.673394500703978, 5.185042253110018, -0.7509567169207, 0, 0, 0, 0, 0, 0;
 -170.2821341374004, -285.2359537322811, -18.02984394798815, 0.1322514847335943, -0.7509567169207013, 24.62999979983148, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
matB: 
[20.99431487687908;
 90.8448330263231;
 -0.971213307257915;
 1.655736040733197;
 -0.4554962466367958;
 -6.108141842074613;
 -0;
 -0;
 -0;
 -0;
 -0;
 -0]
matX: 
[-nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan]
samsdolphin commented 1 year ago

Hi @jun-zhang-cyngn, sorry for my late reply. I am not sure if you have solved this issue. In our latest project here, we choose Eigen to solve the AX=b function. Hope this helps.

yuconglin commented 1 year ago

I came across the same issue.

yuconglin commented 1 year ago

https://github.com/yuconglin/mlcc/commit/e9b0ab1c87fdf161f119cf9e9acc52aae4baf7d8 I fixed in my forked branch. At least it can run through now. It looks like current main branch only support calibrating a single LiDAR's extrinsic.

samsdolphin commented 1 year ago

yuconglin@e9b0ab1 I fixed in my forked branch. At least it can run through now. It looks like current main branch only support calibrating a single LiDAR's extrinsic.

Hi @yuconglin, our extrinsic_refine currently only supports calibrating one extrinsic at once, so if you have multiple LiDARs, you need to execute this launch file multiple times. But our global_refine accepts multiple extrinsics so that you could copy and paste them all together.

yuconglin commented 1 year ago

Thanks for replying. I figured it out too. Thanks again for making the repo public. I learned a lot.