search

agrimgupta92 / sgan

Code for "Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks", Gupta et al, CVPR 2018
MIT License
819 stars 260 forks source link

Question about Evaluation #7

Closed JunweiLiang closed 6 years ago

JunweiLiang commented 6 years ago

Hi, I notice in your code "evaluate_model.py" that you run prediction 20 times for each test example and then use the minimum error as the final number. For other baselines like LSTM and Linear it will produce the same results every time. Do you think it would be fairer to use SGAN to predict 20 times and use the averaged result for evaluation once? Or it is a common practice? Thank you!

agrimgupta92 commented 6 years ago

As we mention in the paper the problem of trajectory prediction is multimodal and it makes more sense to predict multiple possible outcomes given an observed past. Hence, during evaluation we predict multiple paths and then use the best one for evaluation. However, we also report numbers in the paper when we are only drawing 1 sample for a "fair" comparison.