3D object tracking

[beginningfan]

Checklist

• [X] I've read the contribution guidelines.
• [X] I've searched other issues and no duplicate issues were found.
• [X] I've agreed with the maintainers that I can plan this task.

Description

Currently, the tracking module in Autoware is 2D, which only considers the information of the xy plane and cannot adapt to situations such as uphill and downhill. This task aims to design a 3D tracking module to solve the problem of poor 2D performance.

Purpose

Design a 3D tracker to improve tracking performance

Possible approaches

Autoware also has a bytetracker. I will try the effect of this tracker. If it works well, I will transform this tracker into 3D.

Definition of done

• [ ] Testing the existing bytetracker
• [ ] Add a new 3D tracking package
• [ ] Input and output result adaptation to Autoware topics

[beginningfan]

Sorry for the late update. I was unable to test bytetracker because the yolox module could not generate roi. Today I tested the tracking effect of bytetracker. Even when facing fast-moving pedestrians or being temporarily blocked, it still has a relatively good tracking effect. ID switching will only occur when the target's movement direction changes significantly or disappears for a long time. There is the test video Next I will try to modify the tracking logic of bytetracker to 3D and add it to multi_object_tracker

[beginningfan]

Basic tracking functions have been implemented, but there are still some problems.

1. Information such as speed and heading angle has not been added.
2. The truck cannot be tracked (for unknown reasons)

This is a test video. The blue bbox is the detected target and the green bbox is the tracked target.

https://github.com/user-attachments/assets/efc83b69-51da-4f7a-89d5-459b12c38794

[alanmengg]

we have add uuid to track objects, and currently working on solving yaw problem.

https://github.com/user-attachments/assets/2b030322-ca2a-4b53-9cd7-ca99928ff1c5

[alanmengg]

the yaw problem has been partially solved in tracking process, as showed in picture below, orientation of turning object is correct in first picture, as second picture is before yaw correction. still the object orientation will shake if the detection yaw is noisy, we will try to stabilize yaw tracking process.

[alanmengg]

After frame-wise analysis, we discover that yaw is caused by centerpoint detection often give inconsistent yaw between two frames, which has about 180 degree difference. as showed below pitctures(track confidence is draw on left front point) To resolve the problem, we think add ego velocity into tracking process which can estimation absolute velocity of track, then yaw estimation can be inferred from it.

[alanmengg]

There are a lot of low confidence(<0.5) but real object which detected by centerpoint. To track those objects, we modified bytetrack a little by two frame objects association in initilization. As showed in below video, the bus has been tracked. https://github.com/user-attachments/assets/89f8e2e9-912d-4f44-bfd3-7524f047a681

[cyn-liu]

After frame-wise analysis, we discover that yaw is caused by centerpoint detection often give inconsistent yaw between two frames, which has about 180 degree difference. as showed below pitctures(track confidence is draw on left front point) To resolve the problem, we think add ego velocity into tracking process which can estimation absolute velocity of track, then yaw estimation can be inferred from it.

If there is a 180 degree difference in the yaw of the same tracking target between two frames, the algorithm will correct the yaw based on the difference between the observed yaw and the predicted yaw each time the tracking trajectory is updated.