Nuscene Calibration method and Accuracy?

[aryasenna]

Hi everyone,

Today I got in contact with @holger-motional about Nuscene calibration, and was suggested to post here. I'm interested to understand about the camera intrinsic and camera-extrinsic calibration.

In the paper, they are mentioned as so:

Camera extrinsics: We place a cube-shaped calibration target in front of the camera and lidar sensors. The calibration target consists of three orthogonal planes with known patterns. After detecting the patterns we compute the transformation matrix from camera to lidar by aligning the planes of the calibration target. Given the lidar to ego frame transformation computed above, we compute the camera to ego frame transformation.

Camera intrinsic calibration: We use a calibration target board with a known set of patterns to infer the intrinsic and distortion parameters of the camera.

My (starting) questions would be:

1. For camera intrinsic, did you use specific toolbox? e.g. OpenCV toolbox, or Matlab equivalent
2. For camera-lidar extrinsic, what algorithm did you use for the plane alignment?
3. What is the accuracy of your calibration? what metrics did you use (RMSE reproj. error?)

Cheers,

Arya

[whyekit-motional]

Hi @aryasenna, unfortunately the people that worked on the original nuScenes calibration moved on, thus I can only give you my best guesses. Generally, we used our own proprietary codebase and hardware for calibration to ensure reproducibility of the calibration, which was highly accurate. When manually reviewing the settings, it was not possible to improve upon it by eyeballing.

However, for the nuScenes data collection, the calibration was not validated for every car on a daily basis. As such there can be all kinds of disturbances (potholes, violently closing the trunk) that mess up calibration. E.g. in https://github.com/nutonomy/nuscenes-devkit/issues/912 there is a noticeable pitch offset.

[aryasenna]

Hello @whyekit-motional thanks for reply,

I understand the situation, and appreciate your trying to answer.

I assume the calibration method are then not publicly referenceable (e.g. paper?)

Yes of course, dynamic disturbance like pothole is very hard to avoid. It is helpful you pointed the issue where edge case of miscalibration happen

For ideal/typical situation where calibration is good, can you perhaps please comment on the general numerical accuracy? (E.g. in pixel error, and/or degrees and meter error).

It is to understand the quantitative calibration accuracy of nuscene.

Thank you in advance

[whyekit-motional]

@aryasenna yes, the exact calibration method and error are not known. You could analyze the dataset by manually reviewing N samples for each unique calibration, but of course that also deteriorates over time in a way that is hard to model. So you probably want to use the first few logs, not the last ones.