Question on the RGB - Radar alignment issue

brade31919 / radar_depth

Source code of the IROS 2020 paper "Depth Estimation from Monocular Images and Sparse Radar Data"

MIT License

87 stars 13 forks source link

Question on the RGB - Radar alignment issue #6

Closed zzangjinsun closed 3 years ago

zzangjinsun commented 3 years ago

Hello! I have a small question on the RGB and Radar alignment.

I fused RGB and Radar images obtained from tensorboard and got the following results:

Some of images show correct overlap (i.e., projection) results but the right-bottom one looks strange.

Is there any idea on this issue? (Problem of calibration? or synchronization?)

Thank you!

hz658832 commented 3 years ago

do we have imu data which can support to clarify whether the strange projection came from motion disturbance?

LG Haoming

brade31919 commented 3 years ago

Hi @zzangjinsun ,

Can you provide more details about how you generate these plots? The easiest way to verify this is to plot the data returned by the dataloader directly (e.g. data from here). It's possible that this is produced by synchronization issue because they actually didn't have accurate synchronization between LiDAR and Radar in the original nuscenes dataset (They only synced LiDAR and camera). This is less likely to be caused by calibration issue though. I tried the LiDAR and Radar projections myself several times before. If the sensor extrinsics were wrong, we would have encountered this issue on all projects.

Hi @hz658832 , They have IMU data from the original nuscenes database, but I didn't include them into the processed h5 data.

Sincerely, Juan-Ting Lin

zzangjinsun commented 3 years ago

Thank you for your quick reply @brade31919 !

I just overlaid images from the tensorboard but I think those images are the same as ones from the dataloader. I also suspect some synchronization issues of the original nuScenes dataset.

Could you keep this issue open for a while? I will dig into the issue and share the results soon!

zzangjinsun commented 3 years ago

I checked the nuScenes dataset and concluded that there might be a synchronization issue with Radar for a few samples as shown below:

Good sync:

Not sure:

(Vertical offset is observed. Maybe due to a speed bump?)

Overall, I think the calibration quality is quite reasonable.

Thx!

brade31919 commented 3 years ago

Hi @zzangjinsun ,

Thank you for the information! As they described here, it seems that they only handle the synchronization between camera and LiDAR. Therefore, with 13Hz capture frequency, it's quite possible that there exist some synchronization issues between Radar and other sensors.

Sincerely, Juan-Ting Lin