Origin of imuToCamera in calibration.json from vio_record.py?

[jisa]

Hello,

When I run vio_record.py, it also outputs a calibration.json file with a reasonably looking imuToCamera matrix. However, querying the camera (OAK-D W Pro) directly with depthai-python reveals that this camera does not have factory calibrated IMU extrinsics, i.e. there are only zeros in the calibration record area in EEPROM. Where does vio_record.py take the parameters? Is it something I can trust, or should I better calibrate IMU extrinsics myself, for this specific camera?

Regards, Jirka

[oseiskar]

You are right, the EEPROM does not contain the IMU-to-camera matrix in any version of OAK we have tested so far. Therefore we use a fixed "as designed" matrix, which should be "close enough" at least for OAK-D Pro (W), OAK-D S2 and OAK-D, but not calibrated per device individual. You can try replacing it with a more accurately calculated value, but do not expect that to have any noticeable effect on accuracy.

Note that the extrinsic matrix between the cameras (i.e. leftToRight = imuToCamRight * inv(imuToCamLeft)), is read from EEPROM and that is rather accurately calibrated. Stereo rectification also plays a part here. The accuracy of the IMU-to-camera part is not as critical as the accuracy of the stereo camera calibration.

[jisa]

Thank you for confirming and explaining. In this case, I will leave it as is, because the calibration procedure is non-trivial, I do not expect great results and you say there would be little benefit.