AB3DMOT tracker for PointCloud based detection model!

[YoushaaMurhij]

Thanks for the great work! I wonder if it is possible to apply your tracker on 3D detections from point clouds (not images) ? if so, should I change the theta value on the input of the tracker ? Another question, If i have the velocity as a prediction of the model, can I use it to enhance velocity estimation inside kalman filter?

[yrh2526]

Have you solved this problem? I also want to do some works on point cloud, but it seems that the provided only supports image-based tracking

[Po-Jen]

These files already contains the output of 3D object detector: https://github.com/xinshuoweng/AB3DMOT/blob/master/data/KITTI/.

We only care about yaw (which is the theta in the state) because it doesn't make sense for cars to have roll and pitch change.

Maybe also read https://github.com/xinshuoweng/AB3DMOT/issues/53.

[YoushaaMurhij]

Thanks for your response! I tried the tracker on bboxes obtained from detection model on point clouds but the results were not good. I have the same input # h, w, l, x, y, z, theta

[erksch]

@YoushaaMurhij I used this method with a few tweaks for tracking in the Waymo Open Dataset (using the 3D object detections) and the results were great.

[YoushaaMurhij]

Can you please @erksch give some advices about tweaking the method?

[erksch]

@YoushaaMurhij I kind of wonder about your question because the AB3DMOT is made for tracking on 3D upright bounding boxes. If your question is how to gather these boxes from pointcloud data you'll find yourself in the problem of 3D object detection from point clouds. You can start using tracker algorithms once you have the boxes.

I made my repo public at https://github.com/erksch/3D-object-tracking so you could take a look. But it's just for university so I don't guarantee anything. It's very heavily based on this project but a bit cleaned up and enables a comparison between the predictive method with the Kalman-Filter and a naive approach with only prev state overlap association to see if the Kalman-Filter method is actually contributing to better tracking results. You can see my KalmanFilter implementation here. Opposed to the filter in this repo, I used theta velocity in my project, but I actually don't think it has a big impact on the tracking result if you account for angle velocity.

[YoushaaMurhij]

Thanks for your answer @erksch and sorry for the bad explanation. Actually, I have the 3D boxes from 3D object detection model and I am using Hungarian algorithm for tracking and it's results almost good. I tried to enhance it using Kalman-Filter in a similar way to this repository but got worse results.

[erksch]

Just to clarify, using a Kalman-Filter is the "predictive way" (no such thing as a "predictive Kalman-Filter", it's always predictive). I call using a Kalman-Filter the "predictive approach" because it does not match new detections against detections in a previous time frame (what my "naive approach" does), but it matches new detections against the next state prediction that comes from a Kalman-Filter based tracker. I hope that is clear, otherwise please ask :)

[mx2013713828]

Just to clarify, using a Kalman-Filter is the "predictive way" (no such thing as a "predictive Kalman-Filter", it's always predictive). I call using a Kalman-Filter the "predictive approach" because it does not match new detections against detections in a previous time frame (what my "naive approach" does), but it matches new detections against the next state prediction that comes from a Kalman-Filter based tracker. I hope that is clear, otherwise please ask :)

Hello,thanks your repo, I noticed your repo is used for waymo dataset and on segments ,how could I using only detection result and pointcloud to track object? Waiting for your reply~~!