Regarding Odometry Groundtruth

[Ethan-Zhou]

Hi Mathias,

Do you provide the groundtruth trajectories of the left event camera in the dataset? I noticed that an RTK was deployed right there.

Also, will the GT trajectory be available in the CVPRW competition?

Cheers,

yi

[magehrig]

Hi Yi,

We do have RTK measurements (global position measurements) in the GNSS antenna coordinate system. However, these position measurements cannot directly be used as trajectory groundtruth for the cameras because there is a significant translation between the GNSS antenna and the (left) cameras. Unfortunately, there is no workable solution to compute the drift-free camera trajectory at the moment. This is why GT trajectory for the cameras will not be available.

[magehrig]

Reopening this issue because it is a frequently asked question

[xhlinxm]

So...how do you get the odometry that generates the GT value of DSEC-optical flow? If the GPS position is not used, is it obtained by the SLAM algorithm of LiDAR?

[magehrig]

Yes, it's obtained from LIO-Mapping (using Lidar + IMU). It's mentioned in section V of the paper.

[magehrig]

For optical flow gt, a drift-free trajectory is not strictly required because optical flow is computed from a local trajectory.

[xhchen10]

Yes, it's obtained from LIO-Mapping (using Lidar + IMU). It's mentioned in section V of the paper.

Hi Mathias,

I am trying to obtain the trajectory of cameras following your strategy. Where does the IMU data you use in LIO-Mapping come from?

Thanks, Melon

[magehrig]

@xhchen10 It comes from the left event camera. You can find the data on the download page: https://dsec.ifi.uzh.ch/dsec-datasets/download/

We also made some tests a while ago running colmap (on monocular video) on the sequences and it gave quite nice results as well. Maybe there is a way to input stereo to colmap which would give you quite an accurate estimate of the trajectory.

[DogggLee]

Yes, it's obtained from LIO-Mapping (using Lidar + IMU). It's mentioned in section V of the paper.

@magehrig Hi Mathias, I am also trying to get the camera trajectory from the rosbag. But the sequence in lidar_imu.zip doesn't exactly match the training sequences.

For example, rosbag in zurich_city_03 has 544 PointCloud messages, but in training sequence 'zurich_city_03_a', there are only 442 disparity maps here. And the timestamps of disparity maps are not the same as the timestamp of LiDAR in rosbag. How can I determine the position of the first and last frames of the training sequence in the rosbag trajectory?

Thanks, Bin Li

[magehrig]

Hi @DogggLee

Q: For example, rosbag in zurich_city_03 has 544 PointCloud messages, but in training sequence 'zurich_city_03_a', there are only 442 disparity maps here A: I uploaded the raw rosbags we got from data acquisition without post-processing, such as clipping to the specific sequences.

Q: And the timestamps of disparity maps are not the same as the timestamp of LiDAR in rosbag A: The images were captured at 20 Hz while the Lidar was set to 10 Hz for most sequences. We did not enforce that the Lidar scan angle is always the same when an image is captured. The only thing that we enforced is that the internal timestamps of all the sensors are synchronized. This is sufficient to generate depth maps from Lidar odometry because we accumulate depth information over a short period using the ego-motion and Lidar scans.

Q: How can I determine the position of the first and last frames of the training sequences in the rosbag trajectory? A: I think there are two decent options. (1) Use a Lidar(-IMU) odometry method or (2) use a visual odometry or BA method using only video data. I used option (1) as mentioned in the paper. For option (2) we have successfully used Colmap in the past; though we did not get scale because Colmap did not support stereo cameras. You could also use stereo-visual odometry methods that are probably slightly less accurate but should be able to give you good results.

Could you tell me what you're trying to achieve? Is the objective to get a global, drift-free trajectory, or does it only have to be locally accurate?

[DogggLee]

@magehrig Thanks for your reply. I need a global, drift-free trajctory.

I am working on a learning-based visual-LiDAR odometry, but the datasets with longer sequences of synchronized camera and LiDAR data (just like KITTI) are not common. So I attempt to take the disparity map provided in the training sequence as the denser LiDAR data and use them for training and testing. To this end, I need to get the trajectory of each training sequence for evaluation.

I’ve estimated the full trajectory from rosbag by A-LOAM(I tried LIO-Mapping, but its lio-estimator node always died at the begining). Then I need to clip the full trajectory to get the trajectory of the training sequences.

I have tried to determine the start and end frames of each sequence by timestamps, but failed. Take the ‘zurich_city_03_a’ for example, the timestamps of the start/end disparity maps are 57172707635 / 57216807514. The frames with closest timestamps in rosbag are the 66-th and 503-th scans. There are only 438 scans in total, but this sequence has 442 disparity maps. So I must have made some mistakes here. Could you please give me some advice?

[magehrig]

Before going into more depth: You cannot get drift-free trajectories from Lidar+IMU. Are you also planning to somehow take gps data into account? I just want to make sure that what you want is possible before going into more details.

[DogggLee]

Yes, take the GPS in would be better, as the groundtruth poses are usually generated from the RTK. But according to the previous comments, it seems that the RTK trajectory cannot be converted to the camera ordinates?

I just take the LIO estimated trajectory as a pseudo groundtruth for a qualitatively comparison of other odometry methods. So a slight drift is tolerable.

[magehrig]

Ok, I see.

I never enforced that the disparity maps and lidar scans have the same timestamps. To give some context, the goal was to get disparity maps for frame timestamps not the lidar scans. What I did is to interpolate the pose from the lidar odometry at the timestamps when the frames where captured and took the 3D pointcloud to generate the sparse depth/disparity at that timestamp.

If you want to generate depth ground truth specifically at the timestamps when the lidar scan is completed then you might have to use the provided data to generate your own ground truth. A somewhat ugly workaround would be to use the camera intrinsic to project the disparity map to 3D and use your ego-motion estimates from lidar-odometry to project this 3D map at your timestamps.

[DogggLee]

Thanks a lot, it solves my previous doubts. I had previously thought that the camera trajectory of the training sequences are obtained by directly clipping the trajectory estimated from the rosbag data. So I spent a lot of time trying to determine the index of the start frame of the training sequence in the rosbag data.

So can I think that after I get the trajectory of the rosbag data, I need to interpolate the pose of each camera-frame by their timestamps when they are captured to generate the camera trajectory of each train sequence, in addition to project it to the camera coordinates by the extrinsics?

[magehrig]

"I need to interpolate the pose of each camera-frame by their timestamps when they are captured to generate the camera trajectory of each train sequence" A: Yes, if you want to get the poses of the camera at frame times.

"in addition to project it to the camera coordinates by the extrinsics?" A" Not sure what you mean. Can you clarify? I.e., what would you like to achieve?

[DogggLee]

Never mind, I just meant to say that the estimated rosbag trajectory is in LiDAR coordinate system, so it needs to be transformed to the camera coordinate system by the extrinsics between the LiDAR and camera.

Finally, I would like to ask that could you share the config file of LIO-Mapping? There are too many parameters to set here, but I don't really figure out how to set them with this dataset ...

Thanks again for your help.

[magehrig]

These are the files I used for Lio-mapping: https://drive.google.com/file/d/1ERpXIO5EFbVLCk5LzMjLEPPUsQf57Bpo/view?usp=sharing

If I remember correctly I have adapted it sometimes a bit to get better results but it should be a decent start. Interlaken_00 might be a bit tricky because there the Lidar is set to 20 Hz instead of 10 Hz. I think I had to do an ugly hack but I don't remember anymore the details.

[DogggLee]

Thanks a lot!

For the 20Hz sequence, a possible solution is rescale the topic publish frequence into 10Hz by "rosbag play -r 0.5 lidar_imu.bag". But I'm not too sure how this affects the message's timestamp, I'll test it later.