Visualize the 3D coordinate

[herleeyandi]

Hello I am very new in this research. How can we draw the 3D bounding boxes? I have read the paper and still don't understand how to map from our 3D coordinate to the image which is 2D? From your code we can find dimension, center point, also angle, but how can we draw it to the image using opencv? -Thank You-

[yuyijie1995]

1. According to the paper, there are two prediction from our model. The first is the revised angle for each bin of the circle (orient), the second one is the confidence of each bin. So if the orient is 20 degree and the corresponding bin is 120 degree, then the localangle(theta_l in Fig 3)  will be 120+20=40 degree.

2. K[R|T] is for camera itself, not the object. K is the intrinsic, R and T is the rotation (3 DoG) and translation (3 DoG) for camera, which is provided by KITTI. To get the translation of object, you need solve all combination with relation in Section 3.2, but I'm not finishing this part.

3. For the description, you can download the develop kit
   ![image](https://user-images.githubusercontent.com/13130716/42621741-267f1dcc-85f1-11e8-9442-158f164b9926.png)
   The readme file has a full description.

@herleeyandi hi i am sorry to bother you again . I am still confused about the meaning of this sentence "So if the orient is 20 degree and the corresponding bin is 120 degree, then the localangle(theta_l in Fig 3) will be 120+20=40 degree." why the conrresponding bin is 120 degree and 120+20=40?

[yuyijie1995]

Alpha is the observation angle but we don't need to use it. For theta_ray, you can choose the center location of cropped 2d box (x, y). And then you can infer theta_ray, this is also described is this fig

But during my training, I directly use ground truth to infer theta_ray.

" For theta_ray, you can choose the center location of cropped 2d box (x, y)." I do not know how to do it?you don't have depth information from the 2dBOX ,how can you get the theta_ray

[yuyijie1995]

@skhadem I'm still confuse about how to calculate T. We have 4 points of the 2D box, rotation (R), dimension D, intrinsic matrix (K). But how about ith X_o?

How do you get the rotation(R)? In my view,the R is from the global angle ,the red theta in figure3,but in the code, the model only inference the local one ,and the "info['ThetaRay']" we actually don't know, code get the angle by the KITTI labels。 theta = np.arctan2(sin, cos) / np.pi 180 theta = theta + centerAngle[argmax] / np.pi 180 theta = 360 - info['ThetaRay'] - theta " For theta_ray, you can choose the center location of cropped 2d box (x, y)." I am confused about the method . Can you tell me how to do it ? @tonysy @cryax @skhadem @whoiscris

[skhadem]

I think it can be done by using the x,y pixel locations of the center of the box and the pixel width, height of the image. I haven't tried this formula out but it seems like it would work: https://math.stackexchange.com/questions/1320285/convert-a-pixel-displacement-to-angular-rotation

[tjufan]

When I use the location in ground truth and dimension and R_y predicted by model, I can plot a better 3D bounding box in test image. However, I want to plot 3D bounding box without any assitance from ground truth. Then, how can I infer the location or eight corner point for each object ???

[skhadem]

@tjufan I created working version: https://github.com/skhadem/3D-BoundingBox

[MengAjin]

@yuyijie1995 Hi, did you know how to get the theta_ray?

[lucasjinreal]

this package can draw 3d boundbing box on image by providing:

• xyz
• whl
• r_y

and the calibration data

link