search

WeijingShi / Point-GNN

Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud, CVPR 2020.
MIT License
523 stars 114 forks source link

Inference Time and Accuracy on PC #24

Closed abhigoku10 closed 3 years ago

abhigoku10 commented 4 years ago

@WeijingShi thanks for open sourcing the code base i have few queries

  1. what is the inference time fps on the test data
  2. in your paper you have mentioned about " Average precision on downsampled KITTI val. spli" with different scanning line just wanted to know if you re-trained the model when you have reduced the scanning line and validated it
  3. Can this method be used to perform semantic/ panoptic segmentation on point cloud

Thanks for your response in advance

WeijingShi commented 4 years ago

@abhigoku10 thanks for your interest in our work.

  1. The inference time of the 3-layer model in GTX1070 is 643ms per frame. For faster inference, you can modify the config file and replace the content in "runtime_graph_gen_kwargs" with the same as in "graph_gen_kwargs". This boosts the speed significantly and reduces the memory requirement a lot with little accuracy loss.

  2. No, I did not re-trained the model. It's the same model tested by different numbers of scanning lines.

  3. The method is doing well in point classification. And other works on segmentation using GNN/GCN also show promising results. However, the computational cost is a challenging problem. In the current implementation, we have to downsample the point cloud before we construct the graph. Otherwise, the graph will be too large to fit in GPU, and also the running time will be too slow. To get the classification of each input point, maybe a "pyramid" structure is necessary to reduce computational cost.

Thanks,

abhigoku10 commented 4 years ago

@WeijingShi thanks for the response

Q1. thanks for teh pointers will check this out , any idea how much time will get reduced Q2. will try to re-train on the reduced scanning line and check its accuracy on the increased scanning line Q3 yup absolutely right , so any thoughts how to make point gnn to run at real inference time

WeijingShi commented 4 years ago

@abhigoku10 Thanks.

  1. It should be around 250ms per frame in 1070. The modification basically doubles the voxel size during downsampling.
  2. That sounds very interesting. Looking forward to the results.
  3. There are lots of code optimization that can be done on prefetching point cloud / GPU radius neighbor search / parallelizing post-processing. From the network perspective, a 2-layer model seems to have similar accuracy as a 3-layer model, and downsampling the point cloud is also very effective.
EnzeChen1996 commented 4 years ago

@abhigoku10 thanks for your interest in our work.

  1. The inference time of the 3-layer model in GTX1070 is 643ms per frame.

For faster inference, you can modify the config file and replace the content in "runtime_graph_gen_kwargs" with the same as in "graph_gen_kwargs". This boosts the speed significantly and reduces the memory requirement a lot with little accuracy loss.

  1. No, I did not re-trained the model. It's the same model tested by different numbers of scanning lines.

  2. The method is doing well in point classification. And other works on segmentation using GNN/GCN also show promising results. However, the computational cost is a challenging problem. In the current implementation, we have to downsample the point cloud before we construct the graph. Otherwise, the graph will be too large to fit in GPU, and also the running time will be too slow. To get the classification of each input point, maybe a "pyramid" structure is necessary to reduce computational cost.

Thanks,

may I know whether this 643ms is 64 scanning line （without downsample） or not？ Can it run at best performance，within 700ms？ thanks

WeijingShi commented 4 years ago

Hi @EnzeChen1996, Yes, the 643 ms is on 64 scanning line data using the complete T3 model.