search

jchengai / forecast-mae

[ICCV'2023] Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders
https://arxiv.org/pdf/2308.09882.pdf
154 stars 16 forks source link

Probabilities for all agents in multi-agent forecasting #5

Closed KexianShen closed 9 months ago

KexianShen commented 9 months ago

Great work.

I have a question on the probabilities for the trajectory in multi-agent forecasting. Why is probability (pi) with size of (B, K) instead of (B, K, N) like trajectory which is (B, K, N, 60, 2)?

jchengai commented 9 months ago

Hi, @KexianShen. It's modelled as a joint prediction task (as required by the benchmark) where the probability is evaluated at the scene level. (B, K, N) is the marginal prediction case where each agent may have different probability for each mode.

KexianShen commented 9 months ago

@jchengai, thank you for the explanation. Could you give any suggestions on the submission to the argo2 benchmark when my pi has a size of (B, K, N)? Thank you.

The first idea I came up with is ranking the pi, then sorting the trajectories, and finally, only keeping the focal agent's pi.

jchengai commented 9 months ago

Well, it is possible to merge the marginal probability through post-processing tricks such as sorting, averaging et al. and see which one is better through local validation.

However, I'm not sure whether it is worthy spending time on this, as it does not make much sense in principle. Joint modeling and training seems to be a more interesting direction.