search

StanfordASL / Trajectron-plus-plus

Code accompanying the ECCV 2020 paper "Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data" by Tim Salzmann*, Boris Ivanovic*, Punarjay Chakravarty, and Marco Pavone (* denotes equal contribution).
MIT License
664 stars 188 forks source link

Fixed results for ETH/UCY #53

Open sff1019 opened 3 years ago

sff1019 commented 3 years ago

Hi,

Thank you for the wonderful work, and making the code available!

While looking into the code in the eccv2020 branch, I've realized that there is an issue with the derivatives of velocity and acceleration as mentioned in several issues before (issue #26 and issue #40). Unfortunately I could not find results on the fixed version, therefore I have re-ran all models for ETH/UCY using the eccv2020 branch but with the derivative_of() from the master branch for fair comparison with other methods.

Below are the ADE/FDE (best of 20 samples) for Trajectron++ once I fixed the bug: 

before (ADE/FDE) --> after (ADE/FDE)
ETH. : 0.43/0.86 --> 0.67/1.18
Hotel: 0.12/0.19 --> 0.18/0.28
Univ : 0.22/0.43 --> 0.30/0.54
Zara1: 0.17/0/32 --> 0.25/0.41
Zara2: 0.12/0.25 --> 0.18/0.32

Can you please clarify if the numbers seem reasonable? I am sorry if you have already mentioned the updated results somewhere before.

Thank you,

Hana

BorisIvanovic commented 3 years ago

Thank you so much for doing this @sff1019!! Sorry for my delayed response, this is fantastic and saves me the effort of re-running these results myself :)

The numbers seem reasonable to me because:

  1. Their values are close to the original ones.
  2. At least for univ, your values match well with those from another researcher: https://github.com/StanfordASL/Trajectron-plus-plus/issues/40#issuecomment-819522435

Thank you again very much for doing this! I will leave this issue open until I update the arXiv paper, so that others who visit this repository will see these.

zhangpur commented 2 years ago

Hi, @BorisIvanovic, do you have any update results or models for nuscenes dataset?

ArielleZhang commented 1 year ago

Hi @BorisIvanovic , thanks for the great work! I was also trying to reproduce some of the results from the paper (i.e the second row in table5 (b)) which is the robot tag for nuScenes and I am getting different results than that stated in the paper, could you update the results for nuScenes as well? Also was the codebase in the master branch only updated for the derivative_of(), so I don't need to switch to eccv2020 and edit this function to rerun the models? Appreciate your help!

BorisIvanovic commented 1 year ago

I don't need to switch to eccv2020 and edit this function to rerun the models.

That's correct!

ArielleZhang commented 1 year ago

I don't need to switch to eccv2020 and edit this function to rerun the models.

That's correct!

Thanks! Looking forward to see the updated results for nuscenes!

xwasco commented 6 months ago

Thank you again very much for doing this! I will leave this issue open until I update the arXiv paper, so that others who visit this repository will see these.

Hi @BorisIvanovic, are you planning to update the arXiv with the fixed results? It might be helpful for other researchers working in this area. Thanks a lot for your work :) super!