PJLab-ADG / SensorsCalibration

OpenCalib: A Multi-sensor Calibration Toolbox for Autonomous Driving
Apache License 2.0
2.29k stars 545 forks source link

请教一个问题,关于camera 和 lidar标定发散 #95

Open zhaiyuqiang opened 1 year ago

zhaiyuqiang commented 1 year ago

非常好的工具库 目前用在自己的数据集上,有部分失败case,虽然标定失败,但是分数比较高。楼 主能帮忙看看,从哪里排查一下原因吗?感谢🙏

贴一下效果图: error_proj.png image error_seg_proj.png image refined_proj.png image refined_proj_seg.png image

打印的log如下: ` intrinsic: 7389.27 0 1976.89 0 7389.27 1107.01 0 0 1 extrinsic: -0.999908 0.0111884 -0.00772083 0.0346711 0.00734236 -0.0334822 -0.999412 -0.594484 -0.0114403 -0.999377 0.033397 -0.868748 0 0 0 1 dist: 0 dist: 0 dist: 0 dist: 0

N_MASK: 147 Initial error set to (r p y x y z):0 0 0 0 0 0 extrinsic_deltaT: 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ----------Start processing data---------- Plane points: 23606 Plane points: 4074 Plane points: 1529 Plane points < 1500, stop extracting plane. Euclidean cluster number: 23 Extract 26 segments from point cloud. pt.curvaturemax: 0.288433 Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//init_proj.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//init_proj_seg.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//error_proj.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//error_proj_seg.png ----------Start calibration---------- init_score: 0.88492 Start brute-force search around [-5,5] degree and [-0,0] m match score increase to: 0.885669, var:-4 -5 3 0 0 0 match score increase to: 0.889368, var:-4 -5 5 0 0 0 match score increase to: 0.890366, var:-4 3.5 -5 0 0 0 match score increase to: 0.893341, var:-4 4 -5 0 0 0 match score increase to: 0.895034, var:-4 4.5 -5 0 0 0 match score increase to: 0.895734, var:-4 5 -5 0 0 0 match score increase to: 0.896937, var:-3.5 -4 3.5 0 0 0 match score increase to: 0.897148, var:-3.5 -3.5 3.5 0 0 0 match score increase to: 0.897169, var:-3.5 -3 3.5 0 0 0 match score increase to: 0.897367, var:-3.5 0.5 -4.5 0 0 0 match score increase to: 0.897967, var:-3.5 1 -5 0 0 0 match score increase to: 0.900123, var:-3.5 1.5 -5 0 0 0 match score increase to: 0.902913, var:-3.5 2 -5 0 0 0 match score increase to: 0.905038, var:-3.5 2.5 -5 0 0 0 match score increase to: 0.906424, var:-3.5 3 -5 0 0 0 match score increase to: 0.908217, var:-3.5 3.5 -5 0 0 0 match score increase to: 0.908556, var:-3.5 4 -5 0 0 0 match score increase to: 0.909029, var:-3 -0.5 -4.5 0 0 0 match score increase to: 0.909546, var:-3 0 -5 0 0 0 match score increase to: 0.910743, var:-3 0 -4.5 0 0 0 match score increase to: 0.913164, var:-3 0.5 -5 0 0 0 match score increase to: 0.91369, var:-3 0.5 -4.5 0 0 0 match score increase to: 0.91554, var:-3 1 -5 0 0 0 match score increase to: 0.916483, var:-3 1 -4.5 0 0 0 match score increase to: 0.917613, var:-3 1.5 -4.5 0 0 0 match score increase to: 0.918743, var:-3 2 -4.5 0 0 0 match score increase to: 0.919985, var:-2.5 -0.5 -4.5 0 0 0 best var:-2.5 -0.5 -4.5 0 0 0 Start brute-force search around [-0.9,0.9] degree and [-0,0] m match score increase to: 0.920313, var:-0.15 0.45 -0.15 0 0 0 match score increase to: 0.920421, var:-0.15 0.6 -0.15 0 0 0 match score increase to: 0.921194, var:-0.15 0.75 -0.15 0 0 0 match score increase to: 0.921754, var:0 0.75 -0.15 0 0 0 best var:0 0.75 -0.15 0 0 0 Start random search around [-0.5,0.5] degree and [-0.1,0.1] m match score increase to: 0.921827, val:-0.0972841 -0.0319051 -0.0438528 0.00124263 -0.0952072 -0.0672131 match score increase to: 0.922007, val:-0.044532 0.170704 -0.0019033 0.0171676 -0.0917374 0.0199381 match score increase to: 0.92233, val:-0.134128 0.409134 0.0497728 0.0204192 -0.0582624 -0.0402004 match score increase to: 0.922513, val:0.0467862 0.481855 -0.0335861 0.0197454 -0.0602594 0.0982843 best val:0.0467862 0.481855 -0.0335861 0.0197454 -0.0602594 0.0982843 ---------------Result--------------- Error:2.43272 4.69165 -0.853981 0.0645138 -0.136685 0.0848218 Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//refined_proj.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//refined_proj_seg.png Calibration result was saved to file calib_result.txt Total calib time: 149.946s `

xiaokn commented 1 year ago

非常好的工具库 目前用在自己的数据集上,有部分失败case,虽然标定失败,但是分数比较高。楼 主能帮忙看看,从哪里排查一下原因吗?感谢pray

贴一下效果图: error_proj.png image error_seg_proj.png image refined_proj.png image refined_proj_seg.png image

打印的log如下: ` intrinsic: 7389.27 0 1976.89 0 7389.27 1107.01 0 0 1 extrinsic: -0.999908 0.0111884 -0.00772083 0.0346711 0.00734236 -0.0334822 -0.999412 -0.594484 -0.0114403 -0.999377 0.033397 -0.868748 0 0 0 1 dist: 0 dist: 0 dist: 0 dist: 0

N_MASK: 147 Initial error set to (r p y x y z):0 0 0 0 0 0 extrinsic_deltaT: 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ----------Start processing data---------- Plane points: 23606 Plane points: 4074 Plane points: 1529 Plane points < 1500, stop extracting plane. Euclidean cluster number: 23 Extract 26 segments from point cloud. pt.curvaturemax: 0.288433 Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//init_proj.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//init_proj_seg.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//error_proj.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//error_proj_seg.png ----------Start calibration---------- init_score: 0.88492 Start brute-force search around [-5,5] degree and [-0,0] m match score increase to: 0.885669, var:-4 -5 3 0 0 0 match score increase to: 0.889368, var:-4 -5 5 0 0 0 match score increase to: 0.890366, var:-4 3.5 -5 0 0 0 match score increase to: 0.893341, var:-4 4 -5 0 0 0 match score increase to: 0.895034, var:-4 4.5 -5 0 0 0 match score increase to: 0.895734, var:-4 5 -5 0 0 0 match score increase to: 0.896937, var:-3.5 -4 3.5 0 0 0 match score increase to: 0.897148, var:-3.5 -3.5 3.5 0 0 0 match score increase to: 0.897169, var:-3.5 -3 3.5 0 0 0 match score increase to: 0.897367, var:-3.5 0.5 -4.5 0 0 0 match score increase to: 0.897967, var:-3.5 1 -5 0 0 0 match score increase to: 0.900123, var:-3.5 1.5 -5 0 0 0 match score increase to: 0.902913, var:-3.5 2 -5 0 0 0 match score increase to: 0.905038, var:-3.5 2.5 -5 0 0 0 match score increase to: 0.906424, var:-3.5 3 -5 0 0 0 match score increase to: 0.908217, var:-3.5 3.5 -5 0 0 0 match score increase to: 0.908556, var:-3.5 4 -5 0 0 0 match score increase to: 0.909029, var:-3 -0.5 -4.5 0 0 0 match score increase to: 0.909546, var:-3 0 -5 0 0 0 match score increase to: 0.910743, var:-3 0 -4.5 0 0 0 match score increase to: 0.913164, var:-3 0.5 -5 0 0 0 match score increase to: 0.91369, var:-3 0.5 -4.5 0 0 0 match score increase to: 0.91554, var:-3 1 -5 0 0 0 match score increase to: 0.916483, var:-3 1 -4.5 0 0 0 match score increase to: 0.917613, var:-3 1.5 -4.5 0 0 0 match score increase to: 0.918743, var:-3 2 -4.5 0 0 0 match score increase to: 0.919985, var:-2.5 -0.5 -4.5 0 0 0 best var:-2.5 -0.5 -4.5 0 0 0 Start brute-force search around [-0.9,0.9] degree and [-0,0] m match score increase to: 0.920313, var:-0.15 0.45 -0.15 0 0 0 match score increase to: 0.920421, var:-0.15 0.6 -0.15 0 0 0 match score increase to: 0.921194, var:-0.15 0.75 -0.15 0 0 0 match score increase to: 0.921754, var:0 0.75 -0.15 0 0 0 best var:0 0.75 -0.15 0 0 0 Start random search around [-0.5,0.5] degree and [-0.1,0.1] m match score increase to: 0.921827, val:-0.0972841 -0.0319051 -0.0438528 0.00124263 -0.0952072 -0.0672131 match score increase to: 0.922007, val:-0.044532 0.170704 -0.0019033 0.0171676 -0.0917374 0.0199381 match score increase to: 0.92233, val:-0.134128 0.409134 0.0497728 0.0204192 -0.0582624 -0.0402004 match score increase to: 0.922513, val:0.0467862 0.481855 -0.0335861 0.0197454 -0.0602594 0.0982843 best val:0.0467862 0.481855 -0.0335861 0.0197454 -0.0602594 0.0982843 ---------------Result--------------- Error:2.43272 4.69165 -0.853981 0.0645138 -0.136685 0.0848218 Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//refined_proj.png Image saved: data/online_calib_projection_DR7857_20230405094028/DR7857_20230405094028_lidar1281680658901282_732_demotion//calib_res//refined_proj_seg.png Calibration result was saved to file calib_result.txt Total calib time: 149.946s `

我们也发现在部分场景上会误匹配,后续会更新一下新版本的代码,您可以持续关注一下

zhaiyuqiang commented 1 year ago

好的,很期待。 有一个想法,后面是否可以增加类别,比如用Grounded-SAM,这样针对标志牌,车道线,Marker,traffic light可以适当增加优化权重。有些失败场景,算法会关注到栅栏等目标,标志牌反而对齐失败。@xiaokn