search

PRBonn / Mask4D

Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences, RA-L, 2023
MIT License
50 stars 3 forks source link

Questions regarding building a pipeline to work on real-time data #3

Closed himadrir closed 1 month ago

himadrir commented 5 months ago

First of all great work and really appreciate it for making it public. I am looking to design a pipeline where it can work on real-time on LiDARs such as Ouster/Velodyne. I went through the code and it seems it does some batch loading, etc(correct me if I am wrong) to load the data. Could you give me some pointers or give me a high level overview about how do I go about doing this? Thanks!

rmarcuzzi commented 5 months ago

Hi and thanks for your interest in our work!

In this pipeline, during training, we load a sequence of scans and process them one after the other sequentially to train for tracking. During inference, we feed one scan at a time as if we were driving around collecting data sequentially. The network is designed to work with batch size one but the inference time is not suitable for real-time as it is. The network needs around 500ms per scan. You could try reducing the size of the network, trying to use less precision, or optimizing the code to increase runtime.

I hope my answer was useful!