"Real Time" Lidar Segmentation Help

[JordySehn]

Hi there,

I am a researcher who has been working with your code for a few weeks trying to get together a pipeline to feed ROS Lidar point clouds through your network in real-time, and then export the result back to a ROS pointcloud.

After spending quite abit of time putting together a working pipeline, I realised that due to the slow nature of your pointnetize function (about 0.5s processing time), which as far as I can tell is required to compute the neighbours to feed into your network, I dont think real time operation on data which is not pre-processed is possible as you tout in your paper.

Am I missing something here? Or am I just out of luck on this one?

Thanks for your time

[pbias]

Hi,

Thanks for you message. You're not missing anything here, but you have to consider that the given implementation focuses on the network, thus the preprocessing is done offline by creating TFRecords, and is not meant to be used "as-is" for real-time inference. The pointnetize function is fully implemented in python in a really suboptimal way (using loops) in order to clearly outline the way data is preprocessed.

For real-time purpose, I would suggest to code a faster implementation of pointnetize by either vectorizing the preprocessing, or implementing it in a lower level language (for example using Cython).

++

[JordySehn]

Thanks for the quick response!

I suspected as much, that's very fair. After doing some thinking on it I decided to take a crack at replacing the pointnetize function with an equivalent using a purely matrix manipulation approach and this is looking very promising to meet my speed demands. When it is complete and tested I would be happy to share if this would be of interest.

Generally speaking, fantastic work on this! I am excited to get this running on our robot.

[pbias]

Thank you very much ! I'm glad you found a workaround, and I would be definitely interested if you feel like sharing your solution !

Good luck :)