search

rainmaker22 / SMART

[NeurIPS 2024] SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction
Apache License 2.0
76 stars 12 forks source link

How to predict static objects? #15

Open pengzhenghao opened 1 month ago

pengzhenghao commented 1 month ago

As shown in the code here:

https://github.com/rainmaker22/SMART/blob/a329361b63082359be56c9bfaa7e76336c19115f/smart/modules/agent_decoder.py#L465

The heading of the object is inferred from the displacement. However, for static obstacle, how you can get the estimated heading? The heading will be 0 if use arctan2 from displacement.

Also, if the model predict a tiny movement around the static obstacle, the heading will be extremly noisy. How you deal with this? Is there a mechanism to filter out this case?

rainmaker22 commented 1 month ago

https://github.com/rainmaker22/SMART/blob/a329361b63082359be56c9bfaa7e76336c19115f/smart/modules/agent_decoder.py#L464 In fact, according to the code, heading is not calculated based on the displacement between trajectory points. Instead, it's derived from the predicted token’s pose corresponding to the vehicle’s rectangular outline. So even for stationary tokens, the heading remains stable and consistently forward-facing.