search

placeforyiming / IROS21-FIDNet-SemanticKITTI

An extremely simple, intuitive, hardware-friendly, and well-performing network structure for LiDAR semantic segmentation on 2D range image. IROS21
68 stars 11 forks source link

Question about docker #1

Closed DavideCoppola97 closed 3 years ago

DavideCoppola97 commented 3 years ago

hey! I have just discovered your work and I am very interested, I would like to inference with your weights on some of my pointckouds. Given that I have never used a docker I would like to ask you what procedure to follow and what to install to prepare the development environment (or i can directly use the python script without following the previous steps and only installing the requirements library?)(Windows 10 or google collab), thanks

EDIT:I managed to get the python scripts to work, the inference works and the gpu is used correctly. Unfortunately this network has very low performance on my pointclouds, in practice the network assigns label 81 (road sign) to almost all points (obtained with Carla Simulator), I had the same problem also using SalsaNext, SqueezeSegV3 and Darknet53 (this' last is the one that works best as it doesn't put the same id at all points, even if it still has poor performance). The problem from what I understand lies in the geomteria of the pointclouds, I set the simulator parameters to simulate a velodyne hdl64e (the same used to create semanticKITTI) and in fact the point clouds are similar, but despite this the networks are wrong sensationally to make inferences on these points Cattura

placeforyiming commented 3 years ago

hey! I have just discovered your work and I am very interested, I would like to inference with your weights on some of my pointckouds. Given that I have never used a docker I would like to ask you what procedure to follow and what to install to prepare the development environment (or i can directly use the python script without following the previous steps and only installing the requirements library?)(Windows 10 or google collab), thanks

EDIT:I managed to get the python scripts to work, the inference works and the gpu is used correctly. Unfortunately this network has very low performance on my pointclouds, in practice the network assigns label 81 (road sign) to almost all points (obtained with Carla Simulator), I had the same problem also using SalsaNext, SqueezeSegV3 and Darknet53 (this' last is the one that works best as it doesn't put the same id at all points, even if it still has poor performance). The problem from what I understand lies in the geomteria of the pointclouds, I set the simulator parameters to simulate a velodyne hdl64e (the same used to create semanticKITTI) and in fact the point clouds are similar, but despite this the networks are wrong sensationally to make inferences on these points Cattura

From my understand, the input may be the major problem. All eight channels in this paper should be correct, include (x, y, z, r, remission, n1, n2, n3). The (x, y, z, r, n1, n2, n3) are determined by the sensor setting, like where is the LiDAR coordinate. The KITTI put LiDAR on top with x forward, y left and z up. The remission is determined by object materials. You may check if all those things are the same as KITTI.

If you want to test pretrained model, you may further save the generated data from Carla with the same format as KITTI. This will make sure preprocessing steps work well on your own data.

Do you have GPU? I just think the remission may be a challenge for simulators like Carla. So if you can train the model, you may can change the model to exclude remission from the input. Then train on model from the scratch.

DavideCoppola97 commented 3 years ago

Thanks for the answer, as regards the orientation of x, y, z I have already transformed the standard i Carla to adapt it to that of KITTI, as regards the parameter you call `` remission '' I don't really know. if I am simulating it correctly, since carla's lidar is 'perfect', all points are associated with intensity equal to 1, since: The intensity is a function of the material properties of the surfaces, their orientations with respect to the LIDAR sensor, as well as distance and atmospheric conditions. Generally a lower intensity implies the estimate will be less reliable. So it is true that a "perfect" LIDAR signal would be expected to have high intensity for all points, and this is really the case with carla.

placeforyiming commented 3 years ago

Thanks for the answer, as regards the orientation of x, y, z I have already transformed the standard i Carla to adapt it to that of KITTI, as regards the parameter you call `` remission '' I don't really know. if I am simulating it correctly, since carla's lidar is 'perfect', all points are associated with intensity equal to 1, since: The intensity is a function of the material properties of the surfaces, their orientations with respect to the LIDAR sensor, as well as distance and atmospheric conditions. Generally a lower intensity implies the estimate will be less reliable. So it is true that a "perfect" LIDAR signal would be expected to have high intensity for all points, and this is really the case with carla.

I guess so, I think you may visualize all those channels along with the KITTI. There might be some visual differences to make the model perform worse. A simple solution would be training the model from the scratch with input that excludes the intensity. In this project, you can set the remission, range and with_normal all as false, then only the naive x-y-z will be used. The model should be safe, but the performance will be few points lower

placeforyiming commented 3 years ago

Feel free to open it if you want further discussion