OpenCV fisheye distortion coefficients

[tobiascornille]

From the distortPoint method in renderer.py, the distortion coefficients (k1, k2, p1, p2, k3) seem to match the OpenCV camera model. However, OpenCV also has a special distortion model for fisheye cameras. It is seemingly based on the Kannala-Brandt model, which should be better suited for wide-angle cameras, such as the fisheye cameras on your vehicle.

I was wondering if you these distortion coefficients (k1, k2, k3, k4) are available somewhere, or if you have instructions on how we could recalibrate the fisheye cameras.

[TamasMatuszka]

Hi @tobiascornille,

You can find the original calibration files (including distortion coefficients) under the sequence folders, e.g., AIMOTIVE_DATASET/val/urban/20210728-063719-00.08.15-00.08.30@Yoda/sensor/calibration/calibration.json.

The fisheye cameras were calibrated with the Mei model. I copied one fisheye camera example from the calibration file:

"M_FISHEYE_L": { "RT_body_from_sensor": [ [ 0.9985641410429347, 0.016869706719295358, -0.05084357597957023, 1.0571999549865723 ], [ 0.0533535079591374, -0.3982768448676742, 0.9157122681446963, 0.6378999948501587 ], [ -0.004802021620071484, -0.9171101176180838, -0.39860503352429555, 1.4144999980926514 ], [ 0.0, 0.0, 0.0, 1.0 ] ], "RT_sensor_from_body": [ [ 0.9985641410429352, 0.053353507959137424, -0.004802021620071326, -1.0829237078417364 ], [ 0.016869706719295222, -0.3982768448676743, -0.9171101176180838, 1.5334784037272822 ], [ -0.05084357597957034, 0.915712268144697, -0.3986050335242957, 0.033445794263066396 ], [ 0.0, 0.0, 0.0, 1.0 ] ], "custom_vars": { "csi_group_isp_settings_path": "/opt/nvidia/nvmedia/nit/IMX390RGGBE3_200FOV_RGGB.nito", "csi_group_type": "IMX390RGGBE3_200FOV_RGGB_CPHY_x4", "csi_link_id": 5, "misalign_mask_file": "misalign_mask_M_FISHEYE_L.png", "misalign_ref_angle_error": 0.9, "misalign_ref_file": "misalign_ref_M_FISHEYE_L.png", "obstruction_mask_file": "obstruction_masks/M_FISHEYE_L_mask.png" }, "device_id": "TE04913", "distortion_coeffs": [ 0.03364013761086061, -0.08420780755924384, 0, 0, 0 ], "extrinsic_error": -1, "focal_length_px": [ 575.2252458162137, 501.2711423954533 ], "image_resolution_px": [ 1280, 704 ], "label": "M_FISHEYE_L", "model": "mei", "pixel_format": 29, "pos_meter": [ 1.0571999549865723, 0.6378999948501587, 1.4144999980926514 ], "principal_point_px": [ 641.9449779750069, 344.73654890450604 ], "reprojection_error": 0.35321057183345467, "sensor_type": "camera", "transform": "none", "uri": "basler:TE04913", "xi": 1.0222136193840399, "yaw_pitch_roll_deg": [ -93.1780007328259, -23.491001310533154, -0.3000000243565164 ] }

[tobiascornille]

Any reason why the Mei model (for omnidirectional cameras) was used instead of the fisheye camera model from OpenCV?

[TamasMatuszka]

Hi @tobiascornille,

The pinhole camera model can be seen as a special case of the Mei model. This way, one camera model can be used for calibration instead of two.

[haoran15]

为什么使用 Mei 模型（用于全向相机）而不是 OpenCV 的鱼眼相机模型有什么原因？

The Mei model can also be used to calibrate fisheye cameras, and OpenCV's fisheye cameras use the KB model, and when the Fov of the fisheye camera exceeds 180°, the KB model is not applicable.