how to get the same picture(image with bounding box) as yours shown in READ.ME

[kathy-lee]

Hi, thanks for your code sharing. In the code, the coordinate of center point P(x,y,z) and orientation matrix of a ground truth target are calculated and saved, are they in the 3d camera coordinator system? For displaying this bounding box on an image, it still need to be converted to 2d image pixel coordinator system which uses the K matrix, is KP(x,y,z) right? I calculated like this, but the value of the x,y,z turned out extremely big than the size of the image(2048618).

[wzan0001]

Yes, 3d labels in the camera coordinator system. show 3d box in the image can use the p2 matrix.

[kathy-lee]

Thanks for your reply! I tried P2 * P(x,y,z) to convert the object top center point in 3d camera coordinate system to 2d image pixel system, P2 is K matrix : P2: [1.81798103e+03 0 1.04027484e+03 0 0 1.81683987e+03 3.19497539e+02 0 0 0 1 0] , and P(x,y,z) is the top center point of the first object in first file "000035.json" : [ '-8.081753936558147', '0.20091447818807717', '75.59098003244179']. But the multiply result is quite big [63942.9192729 24516.15337888 75.59098 ], I think the first two value should be the x,y pixel cooridnate in 2d image which should be smaller than 2048 and 618, is something wrong in this process? Thanks again!

[wzan0001]

the final pixel is [63942.9192729 24516.15337888 75.59098]./75.59098

[kathy-lee]

Yes, it needs normalization, it works now, thanks a lot!

[kathy-lee]

Hi, I still have two further questions, hope you can help me to make it clear:

1. The position of the top center point is calculated after converting the center point position to the camera coordinator system by adding height/2 to y, but I think height/2 should be added before converting to the camera coordinator system, because this value may change after the converting, and in the document of Astyx dataset in section 3.7 of groundtruth data it says: "center3d is 3D object position x,y,z(center of 3D bounding box) in master sensor coordinate system" which I think master sensor means radar sensor, I am not sure about this.
2. The yaw_angle is calculated from orientation_quat and Tr_radar_to_cam, it picks out only the rotation in yaw axis. If don't separate out the yaw_angle but directly use the matrix radar_quat_to_mat(matrix converted from orientation_quat) to do projection like this: KTr_radar_to_camradar_quat_to_mat*xyz_box, is this wrong?

[wzan0001]

1. I think the master sensor is radar. Maybe it's better to add height/2 before, you are right.
2. I mainly focused on the yaw_angle, just extracted it.

[kathy-lee]

Many thanks for your reply!

[ZhuyunZhou]

@kathy-lee Hello, you seem to have a wonderful version with normalization and other encountered problems. Could you please send your improved version to me? My email address is judyzhouzy@yeah.net. Thanks a lot!

[kathy-lee]

Hi @ZhuyunZhou, for convenience I paste my code snippet here, you can replace the corresponding code in radar_label_convert_kitti_format.py file:

centerpoint=np.array(obj['center3d'])
centerpoint=np.reshape(centerpoint,(1,3))
top_centerpoint = centerpoint
top_centerpoint[:,1] = top_centerpoint[:,1] + dim[2]*0.5
camera_centerpoint = utils.radar_to_cam_frame(top_centerpoint, frame_calib)
anotation.append(str(camera_centerpoint[0][0]))
anotation.append(str(camera_centerpoint[0][1]))
anotation.append(str(camera_centerpoint[0][2]))

The only thing I did is getting top center point pos before converting to camera coord, because the ground truth bounding box is defined in radar coord. Hope this can help.

[ZhuyunZhou]

@kathy-lee Thanks a lot!