Inference code to evaluate one pointcloud file on Nuscnes dataset

[ghost]

Hi,

I trained the code for Nuscenes dataset for all classes using all.fhd.config,all.pp.largea.config,

But with the checkpoints that i got for training, i tried to inference part on single pointcloud data file, code was getting crashed.

Kindly let us now how to do inference on single pointcloud files rather than multi-sweeps. Kindly help and any help would be really appreciated.

[ghost]

@poodarachu, @traveller59 Can you kindly help. Do the needful

[ghost]

Do you want me to retrain with create_data with sweeps=1 in order to work with individual pointcloud for inference? Kindly reply

[ghost]

I ran the inference code with trained models all.fhd.config and all.largea.config models for one point cloud. I'm getting too many false positives with score threshold of 0.3,0.4. but my precision and recall graphs performed well. Kindly suggest please help

[ghost]

[ghost]

Kindly suggest as the precision and recall curves are increasing only. Still for one pointcloud iam getting too many false-positives.

[ghost]

@poodarchu, can u kindly suggest please

[ghost]

sample result for one point cloud. This ia all.fhd.config trained upto voxelnet-64515.tckpt. Kindly suggest how to go ahead for improving.

[ghost]

I took the base code from traveller59 github without any modifications in the model/code. But detections are far from true positives. Kindly suggest. I need to retrain? If retrain what parameters i need to modify to get decent results on single pointcloud.

[ghost]

In the github all.fhd.config, nms_score_threshold: 0.05 nms_iou_threshold: 0.5.. Are you talking about these parameters? These are the params or some other params? Kindly suggest

[ghost]

I used simple_inference.py to evaluate the single_pointcloud_file with visualization using mayavi for 3D Even i used bird eye view results also but that also bad. I used the code below.

config_path = "/home/ubuntu/LIDAR/traveller59_2/second/configs/nuscenes/all.fhd.config" config = pipeline_pb2.TrainEvalPipelineConfig() with open(config_path, "r") as f: proto_str = f.read() text_format.Merge(proto_str, config) input_cfg = config.eval_input_reader model_cfg = config.model.second config_tool.change_detection_range(model_cfg, [-50, -50, 50, 50]) device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

ckpt_path = "/home/ubuntu/AWS/allfhd/voxelnet-64900.tckpt" net = build_network(model_cfg).to(device).eval() net.load_state_dict(torch.load(ckpt_path)) target_assigner = net.target_assigner voxel_generator = net.voxel_generator

grid_size = voxel_generator.grid_size feature_map_size = grid_size[:2] // config_tool.get_downsample_factor(model_cfg) feature_map_size = [*feature_map_size, 1][::-1]

anchors = target_assigner.generate_anchors(feature_map_size)["anchors"] anchors = torch.tensor(anchors, dtype=torch.float32, device=device) anchors = anchors.view(1, -1, 7)

v_path = "n008-2018-05-21-11-06-59-0400__LIDAR_TOP__1526915243047392.pcd.bin" scan = np.fromfile(v_path, dtype=np.float32) points = scan.reshape((-1, 5))[:, :4] res = voxel_generator.generate(points, max_voxels=90000)

voxels, coords, num_points = voxel_generator.generate(points, max_voxels=90000)

print(res['voxels'].shape)

add batch idx to coords

res['coordinates'] = np.pad(res['coordinates'], ((0, 0), (1, 0)), mode='constant', constant_values=0) res['voxels'] = torch.tensor(res['voxels'], dtype=torch.float32, device=device) res['coordinates'] = torch.tensor(res['coordinates'], dtype=torch.int32, device=device) res['num_points_per_voxel'] = torch.tensor(res['num_points_per_voxel'], dtype=torch.int32, device=device)

example = { "anchors": anchors, "voxels": res['voxels'], "num_points": res['num_points_per_voxel'], "coordinates": res['coordinates'], } pred = net(example)[0] boxes_lidar = pred["box3d_lidar"].detach().cpu().numpy() scores = pred['scores'].detach().cpu().numpy() label_preds = pred['label_preds'].detach().cpu().numpy() fig = mlab.figure(figure=None, bgcolor=(0, 0, 0), fgcolor=None, engine=None, size=(1000, 500)) draw_lidar(points, fig=fig) iLen = boxes_lidar.shape[0] for iIndex in range(iLen): score = scores[iIndex] if(score<0.4): continue x = boxes_lidar[iIndex][0] y = boxes_lidar[iIndex][1] z = boxes_lidar[iIndex][2] l = boxes_lidar[iIndex][3] w = boxes_lidar[iIndex][4] h = boxes_lidar[iIndex][5] ry = boxes_lidar[iIndex][6] x_corners = l / 2 np.array([1, 1, 1, 1, -1, -1, -1, -1]) y_corners = w / 2 np.array([1, -1, -1, 1, 1, -1, -1, 1]) z_corners = h / 2 * np.array([1, 1, -1, -1, 1, 1, -1, -1]) corners = np.vstack((x_corners, y_corners, z_corners)) R = rotz(ry) corners = np.dot(R, corners) corners[0, :] = corners[0, :] + x corners[1, :] = corners[1, :] + y corners[2, :] = corners[2, :] + z corners_3d = corners.T color = (0,1,0) line_width = 1 mlab.points3d(x,y,z, color=(1, 1, 1), mode='sphere', scale_factor=0.4) for k in range(0, 4): i, j = k, (k + 1) % 4 mlab.plot3d([corners_3d[i, 0], corners_3d[j, 0]], [corners_3d[i, 1], corners_3d[j, 1]], [corners_3d[i, 2], corners_3d[j, 2]], color=color, tube_radius=None, line_width=line_width, figure=fig) i, j = k + 4, (k + 1) % 4 + 4 mlab.plot3d([corners_3d[i, 0], corners_3d[j, 0]], [corners_3d[i, 1], corners_3d[j, 1]], [corners_3d[i, 2], corners_3d[j, 2]], color=color, tube_radius=None, line_width=line_width, figure=fig) i, j = k, k + 4 mlab.plot3d([corners_3d[i, 0], corners_3d[j, 0]], [corners_3d[i, 1], corners_3d[j, 1]], [corners_3d[i, 2], corners_3d[j, 2]], color=color, tube_radius=None, line_width=line_width, figure=fig)

mlab.show()

[ghost]

Shall i use 10 sweeps and use the smae inference code to verify result? Kindly suggest.