search

woven-planet / l5kit

L5Kit - https://woven.toyota
https://woven-planet.github.io/l5kit
857 stars 278 forks source link

Urban Driver model evaluation #359

Open nlimotional opened 2 years ago

nlimotional commented 2 years ago

Thank you for open source the Urban Driver model. Very interesting paper. Could you please share the details on the evaluation protocol used in the paper? Based on the paper, the evaluation is done by unrolling the 25s scene in the validation dataset. Following the evaluation notebook, I am seeing very different results using the released pretrained models. I think I missed some important evaluation details. I would appreciate if you can provide more details on this.

perone commented 2 years ago

Hi @nlimotional, thanks for the interest. @blazejosinski do you have more info on that ?