question about the directory of 'matrics'

[xuzekai1997]

hello. I want to do somethings about stero images,such as project radar instances to images. But i don't konw matrics that will be used in projection,is it saved in directory 'matrics'? Thank you very much.

[ZhangAoCanada]

Hi,

Theoretically, to project the radar instances onto the camera frame, the best way is to first convert those radar instances to point cloud. Then, you can project those pcl to the camera frame. While saying so, I think it will be a little hard to implement. The reason is, the image plane is a [x, y] 2D plane while the radar instances are organized in [x, z] 2D plane. You may need to figure out a way to project the points from [x, z] to [x, y].

[xuzekai1997]

Hi, I saw that you project camera instances to radar frames in your paper,just like figure 2. So,could you share about the equations about 'camera to radar'.As far as I know,the equation can be figured out after calibration. Thank you.

[ZhangAoCanada]

yes, there is registration matrix, and it is shared at GD. Specifically, the path to registration matrix is sensor_para/registration_matrix.

This matrix is a 2*4 sized matrix. It can directly project stereo pcl [x, y, z, 1] to radar points [x, y].

Hope this is what you are looking for.

Let me know.

[xuzekai1997]

Thanks for your answer. But it i thought if we want to get the data of stereo pcl, we should first project camera bounding boxes to 3D cartisen coordinate system. So could you tell me about the meaning of these matrices in the following picture? Thank you.

[ZhangAoCanada]

For the screenshot you showed me, left_maps.npy and right_maps.npy are derived from cv2.initUndistortRectifyMap(...); all other matrices are derived from cv2.stereoRectify(...).

To the stereo pcl, I didn't use bounding box based detection. Instead, I applied Mask-RCNN to the images before stereo depth estimation. Then, each pixel in the mask prediction is processed through stereo 3D estimation, and thus I could get stereo pcl.