Parsing the rosbag

[midjji]

There are a number of questions I haven't found an answer too in the website: And parsing the bag risks only confirming incorrect assumptions.

Camera Related:

1. What is the topic for the HW trigger times? or is it in the image message somehow? Elaborating a bit, the documentation says the header of the message, but these differ for the cameras and look like time received, not time taken
2. Are the exact same cameras and rig used for all sequences?
3. Are the extrinsics the same for all sequences? Usually they change subtly as the cameras shift and move. If so are they in the corresponding bag? Do they ever change during a recording?
4. Are the cameras global shutter? if not what are the rs readout times and exposure times.
5. What camera model was used, is it the cv::fisheye one?
6. Are there different cameras or is there just one?
7. What is/are the camera bit depth?
8. What is the camera resolution?
9. Are the cameras RGB or grey?
10. Where is the focus placed at? infinity?
11. is the topic image_raw the raw sensor values or has there been some kind of filter, brightness adjustment, gamma or other nonsense? If so whats the inverse function?
12. Are the lenses fixed, or do they change during or between sequences?
13. What is the unit for the extrinsics? i.e. tf_static?
14. What order is your quaternion in ? i.e. real, imag, or imag,real?
15. What is the transform directly i.e. is P_imu_to_cam from cam coordinates to imu coordinates?
16. Do the cameras output frameid, or is there any way to know if there were silent framedrops in the driver before ros got to it?
17. How do I get the camera set frameid from the rosmessages?
18. Is the exposure adaptive or fixed?
19. Are the calibration images available, in case I want to try a different lens model? Specifically one with fewer coefficients.
20. More information about exposure and any adaptive system in the sensor itself?

Time Synchronization

1. Is HW trigger used for the cameras?
2. What is the time for the imus?
3. Whats the time representation, is it float128 seconds (long double on linux), or something problematic with unit, wrapping and overflow issues? it seems like uint64 s + uint64 ns
4. Will the time representation wrap around?
5. How are jumps caused by the p2p handled? They arent

IMU:

1. Are the imu bias calibrated? has this adjustment been done in preprocessing?
2. Are the imu temperature calibrated? has this adjustment been done in preprocessing?
3. The imu seems to be in the format of m/s² and w(t), what is the unit for the latter?
4. The use of a sane format like float m/s^2 for the imu means its probably preprocessed, as imu tend to give out 16bit ints with some selected range usually in g(which is usually assumed to be 10, but is incoherent and inconsistent). Was that range sensibly chosen?

Dataset,

1. is there a non ros dependent dataset to download?
2. is there a standard parser in python or C++, e.g. a simple visualizer, if not I will post one once I get it working.
3. Are there any framedrops? - yes
4. Are there any partial framedrops? - yes
5. Are all framesets complete, i.e. is there ever a missing frame, e.g. the beginning or end of a recording. yes
6. Where are the hashes needed to validate successful download of the bags?
7. Is there a torrent (Better alternative for download as it automatically validates using hashes in chunks, and all other good stuff)

Evaluation

1. Is there a sample validation script available which allows the user to verify the pose formats on the sequences with GT before submission? This eliminates the common but sad mistake when some user uploads a perfectly good result but accidentally swaps quaternion representation or uploads P_base_platform, instead of P_platform_base. Seems like in this repo

I'll add additional questions as I try to implement the dataset parser and visualizer etc.

[hemi86]

Camera Related:

What is the topic for the HW trigger times? or is it in the image message somehow? Elaborating a bit, the documentation says the header of the message, but these differ for the cameras and look like time received, not time taken

The cameras are triggered at the same time and are timestamped in the middle of the exposure. The sync. accuracy with the alphasense/imu and the adis imu is below 100 us. You might also find their documentation and the datasheet useful. The timestamp when the image was taken is is in the header of the message.

Are the exact same cameras and rig used for all sequences?

Yes

Are the extrinsics the same for all sequences? Usually they change subtly as the cameras shift and move. If so are they in the corresponding bag? Do they ever change during a recording?

Mechanical stress and temperature has an influence on the extrinsic calibration. We calibrated the device only once before the recording of the datasets (which btw were recorded at different days).

Are the cameras global shutter? if not what are the rs readout times and exposure times.

Yes, they are global shutter cameras (Sony IMX-273). Please also have a look at the datasheet

What camera model was used, is it the cv::fisheye one?

We used the Pinhole with equidistant distortion model, you can for example use the image undistortion node to undistort the images.

Are there different cameras or is there just one?

They are all the same model.

What is/are the camera bit depth? What is the camera resolution? Are the cameras RGB or grey?

We use the Sony IMX-273 camera sensor.

Where is the focus placed at? infinity?

Yes

is the topic image_raw the raw sensor values or has there been some kind of filter, brightness adjustment, gamma or other nonsense? If so whats the inverse function?

It is the raw data.

Are the lenses fixed, or do they change during or between sequences?

Same as with calibration, the lenses can slightly change due to mechanical stress or temperature.

What is the unit for the extrinsics? i.e. tf_static?

It should be in meters

What order is your quaternion in ? i.e. real, imag, or imag,real?

The format in the calibration file is XYZW

Do the cameras output frameid, or is there any way to know if there were silent framedrops in the driver before ros got to it?

This should be visible in the timestamp.

How do I get the camera set frameid from the rosmessages?

Can you explain what you mean by that?

Is the exposure adaptive or fixed?

It is adaptive.

Are the calibration images available, in case I want to try a different lens model? Specifically one with fewer coefficients.

No the calibration images are not available

More information about exposure and any adaptive system in the sensor itself?

Please see the datasheet of the sensor.

[hemi86]

Time Synchronization

Is HW trigger used for the cameras?

Yes

What is the time for the imus?

The cameras, the alphasense/imu and the alphasense/imu_adis are timestamped using the same clock.

Whats the time representation, is it float128 seconds (long double on linux), or something problematic with unit, wrapping and overflow issues? it seems like uint64 s + uint64 ns

Please see the ROS documention

Will the time representation wrap around?

Yes

How are jumps caused by the PTP handled?

They are not handled.

IMU:

Are the imu bias calibrated? has this adjustment been done in preprocessing? Are the imu temperature calibrated? has this adjustment been done in preprocessing? The imu seems to be in the format of m/s² and w(t), what is the unit for the latter? The use of a sane format like float m/s^2 for the imu means its probably preprocessed, as imu tend to give out 16bit ints with some selected range usually in g(which is usually assumed to be 10, but is incoherent and inconsistent). Was that range sensibly chosen?

We did not perform any postprocessing on the imu data (except for converting it to the ROS IMU message) . I do not know which range sensibility was configured on the imus.

Dataset;

is there a non ros dependent dataset to download? is there a standard parser in python or C++, e.g. a simple visualizer, if not I will post one once I get it working.

You can use for example kalibr_bag_extractor to extract the IMU and image information.

Where are the hashes needed to validate successful download of the bags?

We do not provide hashes at the moment.

Is there a torrent (Better alternative for download as it automatically validates using hashes in chunks, and all other good stuff) Evaluation

No

[midjji]

Thanks,

The IMX-273 sensor supports 8,10, 12 bit depths, which setting was used?

The IMX-273LQR is color, The IMX-273LLR is greyscale. Both support grey output, which one is it, and was color used?

IMX-273 has 1456x1088 resolution, including the blackedout, so the 1440x1080 we got parsing the rosbag makes sense. However, the IMX-273 and its housings support binning. Was binning used?

Parsing the rosbags we got 1440x1080 at 8 bit resolution.

Automatic exposure was used, was automatic gain also used?

Are camera dark, bright calibration maps available? i.e. the images which were taken at full bright and full dark? these are used to remove and interpolate dead and zombie/always saturated pixels, and also to calibrate sensor noise. (Not generally important, just a nice feature)

[hemi86]

The IMX-273 sensor supports 8,10, 12 bit depths, which setting was used?

8bit

The IMX-273LQR is color, The IMX-273LLR is greyscale. Both support grey output, which one is it, and was color used?

The monochrome sensor IMX-273LLR was used

IMX-273 has 1456x1088 resolution, including the blackedout, so the 1440x1080 we got parsing the rosbag makes sense. However, the IMX-273 and its housings support binning. Was binning used?

no binning was performend

Parsing the rosbags we got 1440x1080 at 8 bit resolution.

Automatic exposure was used, was automatic gain also used?

no auto gain was used

Are camera dark, bright calibration maps available? i.e. the images which were taken at full bright and full dark? these are used to remove and interpolate dead and zombie/always saturated pixels, and also to calibrate sensor noise. (Not generally important, just a nice feature)

no, unfortunately no dark/bright calibration maps are available