search

jchengai / forecast-mae

[ICCV'2023] Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders
https://arxiv.org/pdf/2308.09882.pdf
154 stars 16 forks source link

What is the benefit of using a learnable lane type embedding rather than extracting it from data? #3

Closed shengchao-y closed 9 months ago

shengchao-y commented 9 months ago

Hello, thanks for this great project! I have a question regarding the lane_type_embed in forecast-mae/src/model/model_forecast.py. What is the benefit of using a learnable lane type embedding rather than extracting it from data?

https://github.com/jchengai/forecast-mae/blob/d3b80cfba304a01ba26e8b9a99e1b572e9fdd81a/src/model/model_forecast.py#L121C27-L121C47

jchengai commented 9 months ago

Hi, @shengchao-y. Since different sources of inputs (e.g., history, lane, future) are mixed together as the input of the transformer encoder, the learnable type embeddings are intended to differ them.

What is the benefit of using a learnable lane type embedding rather than extracting it from data?

Though I have not tried it, I assume it won't make much difference.

shengchao-y commented 9 months ago

It is interesting, that you use the same type embedding for all types of lanes and it works well...