Visualizing the results

[norbertmarko]

Hello!

Awsome work on the theory, I aslo tried out the repo, works well. My problem is that I can't seem the find guidance on visualizing the 3D bounding box results. Can you give me some hints on this issue? I found the output txt files, and a draw_3D_box() function, but it requires corners and I'm not sure about the whole process.

Thanks in advance!

[Owen-Liuyuxuan]

I was working on some other stuff :(. I copied some codes from a notebook, there could be some bug but I hope it can help you out. Basically the draw_3D_box() function assume the points' order are the same as the BBox3dProjector.

from visualDet3D.networks.utils import BBox3dProjector, BackProjection
def denorm(image):
    new_image = np.array((image * cfg.data.augmentation.rgb_std +  cfg.data.augmentation.rgb_mean) * 255, dtype=np.uint8)
    return new_image

projector = BBox3dProjector().cuda()
backprojector = BackProjection().cuda()

data = dataset[index]
collated_data = dataset.collate_fn([data])
image, P2 = collated_data[0], collated_data[1]
img_batch = image.cuda().float().contiguous() #[1, 3, H, W] Augmented
P2 = torch.tensor(P2).cuda().float() # [1, 3, 4] Augmented

scores, bbox, obj_index = module([img_batch, P2]) # test_forward

img = image.squeeze().permute(1, 2, 0).numpy()
rgb_image = denorm(img) # np.uint8 

bbox_2d = bbox[:, 0:4]
bbox_3d_state = bbox[:, 4:] #[cx,cy,z,w,h,l,alpha]
bbox_3d_state_3d = backprojector(bbox_3d_state, P2.cuda()) #[x, y, z, w,h ,l, alpha]
abs_bbox, bbox_3d_corner_homo, thetas = projector(bbox_3d_state_3d, P2[0]) # check the docstring

for box in bbox_3d_corner_homo:
    box = box.cpu().numpy().T
    rgb_image = draw_3D_box(rgb_image, box)

plt.imshow(rgb_image) # the bboxes are drawed on the augmented image

[norbertmarko]

I was working on some other stuff :(. I copied some codes from a notebook, there could be some bug but I hope it can help you out. Basically the draw_3D_box() function assume the points' order are the same as the BBox3dProjector.

from visualDet3D.networks.utils import BBox3dProjector, BackProjection
def denorm(image):
    new_image = np.array((image * cfg.data.augmentation.rgb_std +  cfg.data.augmentation.rgb_mean) * 255, dtype=np.uint8)
    return new_image

projector = BBox3dProjector().cuda()
backprojector = BackProjection().cuda()

data = dataset[index]
collated_data = dataset.collate_fn([data])
image, P2 = collated_data[0], collated_data[1]
img_batch = image.cuda().float().contiguous() #[1, 3, H, W] Augmented
P2 = torch.tensor(P2).cuda().float() # [1, 3, 4] Augmented

scores, bbox, obj_index = module([img_batch, P2]) # test_forward

img = image.squeeze().permute(1, 2, 0).numpy()
rgb_image = denorm(img) # np.uint8 

bbox_2d = bbox[:, 0:4]
bbox_3d_state = bbox[:, 4:] #[cx,cy,z,w,h,l,alpha]
bbox_3d_state_3d = backprojector(bbox_3d_state, P2.cuda()) #[x, y, z, w,h ,l, alpha]
abs_bbox, bbox_3d_corner_homo, thetas = projector(bbox_3d_state_3d, P2[0]) # check the docstring

for box in bbox_3d_corner_homo:
    box = box.cpu().numpy().T
    rgb_image = draw_3D_box(rgb_image, box)

plt.imshow(rgb_image) # the bboxes are drawed on the augmented image

I was able to figure out a way to show the results with the code you gave me, thank you very much for your help!

[mertmerci]

@norbertmarko Could you please explain the method you figured out to show the results? Thank you in advance!