About box visualization

[irmuun20]

I want to test your model on the sample bin data you provided in the repo. I am trying to visualize the prediction boxes using open3d library but boxes seem incorrect such that person box in blue color is lying horizontally like car while car boxes in red color is displayed correctly. I used the following code to visualize results:

annotation_file = open('result/seq_0_frame_100.bin.txt', 'r')
boxes = []
lines = annotation_file.readlines()
for line in lines:
    tokens = line.split()
    x = float(tokens[0])
    y = float(tokens[1])
    z = float(tokens[2])
    h = float(tokens[3])
    w = float(tokens[4])
    l = float(tokens[5])
    velX = float(tokens[6])
    velY = float(tokens[7])
    theta = float(tokens[8])
    score = float(tokens[9])
    cls = int(tokens[10])
    if score > 0.2:
        boxes.append([x,y,z,h,w,l,theta,cls])

        #box = [h,w,l,x,y,z,rot]

def roty(t):
    """
    Rotation about the y-axis.
    """
    c = np.cos(t)
    s = np.sin(t)
    return np.array([[c, 0, s],
                     [0, 1, 0],
                     [-s, 0, c]])

def box_center_to_corner(box):
    translation = box[0:3]
    h, w, l = box[3], box[4], box[5]
    #if the angle value is in radian then use below mentioned conversion
    #rotation_y = box[6]
    #rotation = rotation_y * (180/math.pi)                             #rad to degree
    rotation = box[6]

    # Create a bounding box outline if x,y,z is center point then use defination bounding_box as mentioned below
    bounding_box = np.array([
        [-l/2, -l/2, l/2, l/2, -l/2, -l/2, l/2, l/2],
        [w/2, -w/2, -w/2, w/2, w/2, -w/2, -w/2, w/2],
        [-h/2, -h/2, -h/2, -h/2, h/2, h/2, h/2, h/2]])

    # Standard 3x3 rotation matrix around the Z axis
    rotation_matrix = np.array([
        [np.cos(rotation), -np.sin(rotation), 0.0],
        [np.sin(rotation), np.cos(rotation), 0.0],
        [0.0, 0.0, 1.0]])

    # Repeat the [x, y, z] eight times
    eight_points = np.tile(translation, (8, 1))

    # Translate the rotated bounding box by the
    # original center position to obtain the final box
    corner_box = np.dot(rotation_matrix, bounding_box) + eight_points.transpose()

    return corner_box.transpose()

    entities_to_draw = []

for box in boxes:
    boxes3d_pts = box_center_to_corner(box)
    boxes3d_pts = boxes3d_pts.T
    boxes3d_pts = o3d.utility.Vector3dVector(boxes3d_pts.T)
    box3d = o3d.geometry.OrientedBoundingBox.create_from_points(boxes3d_pts)
    if box[-1] == 0:
        box3d.color = [1, 0, 0]           #Box color would be red box.color = [R,G,B]
    elif box[-1] == 1:
        box3d.color = [0, 0, 1]
    else:
        box3d.color = [0, 1, 0]
    entities_to_draw.append(box3d)

I will appreciate it if you provide a feedback on this issue. Thank you.

[HaohaoNJU]

That's wired, can you check your det result in the .txt file to see if lwh is reasonable ?

[irmuun20]

It seems like I was using a wrong method to translate rotation. The following fixes the issue. Thank you for your wonderful implementation.

entities_to_draw = []
for box in boxes:
    center_point = box[0:3]
    h, w, l = box[3], box[4], box[5]

    rotation = box[6]

    # Standard 3x3 rotation matrix around the Z axis
    rotation_matrix = np.array([
        [np.cos(rotation), -np.sin(rotation), 0.0],
        [np.sin(rotation), np.cos(rotation), 0.0],
        [0.0, 0.0, 1.0]])

    box3d = o3d.geometry.OrientedBoundingBox(center_point, rotation_matrix, [h,w,l])

    if box[-1] == 0:
        box3d.color = [1, 0, 0]           #Box color would be red box.color = [R,G,B]
    elif box[-1] == 1:
        box3d.color = [0, 0, 1]
    else:
        box3d.color = [0, 1, 0]
    entities_to_draw.append(box3d)

[irmuun20]

Now rotation seems in inverse.

[irmuun20]

is this correct data output from centerpoint? rotation is relative to z axis? [center_x, center_y, center_z, rotation_z, length, width, height]

[HaohaoNJU]

No，the output format in each line is x,y,z,l,w,h,vx,vy,rot,score, class, check here

I also provide visualization code, you can take it as reference ~

[irmuun20]

I checked that you output radian value for rotation as follows: https://github.com/HaohaoNJU/CenterPoint/blob/7728be92e118e633602b8c2d96a5f955d03dec99/src/postprocess.cpp#L194 but when I convert it into degree, it yields still incorrect results degree = radian*(180/pi)

[irmuun20]

Finally fixed the issue by setting rotation to negative value:

entities_to_draw = []
for box in boxes:
    center_point = box[0:3]
    h, w, l = box[3], box[4], box[5]

    rotation = -box[6]

    # Standard 3x3 rotation matrix around the Z axis
    rotation_matrix = np.array([
        [np.cos(rotation), -np.sin(rotation), 0.0],
        [np.sin(rotation), np.cos(rotation), 0.0],
        [0.0, 0.0, 1.0]])

    box3d = o3d.geometry.OrientedBoundingBox(center_point, rotation_matrix, [h,w,l])

    entities_to_draw.append(box3d)

[mariamsamiiiir]

Helloo ,
Can you send me the script you have used for visualization , I try to test on ouster lidar data and I have seen that you have done the same so can you please provide me with the script of visualization and how you have handled intensity and elongation values Thank you in advance