search

XiaolongTang23 / HPNet

[CVPR 2024] HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attention
Apache License 2.0
96 stars 11 forks source link

How is the test conducted? #3

Closed ntuwork closed 3 months ago

ntuwork commented 3 months ago

Thank you for making your code public!

I have a question about how to do testing for interaction dataset. For the interaction dataset, is it necessary to go to the official leaderboard to upload for multi-agent testing and get the results? Is it feasible to test offline? I have observed that many methods do not perform offline testing and do not provide code for offline testing. I would like to know what are the reasons for this.

Thanks a lot.

XiaolongTang23 commented 3 months ago

Thank you for reaching out and for your interest in the code!

Testing indeed requires generating prediction results which then need to be submitted to the official leaderboard. This is an essential step for evaluation and comparison against other methods. We are unable to perform offline testing primarily because the test set does not come with ground truth values accessible to us. This limitation means that to obtain accurate and comparable results, submission to the leaderboard, which has access to the ground truths, is necessary.

I hope this clarifies your concerns. Please feel free to reach out if you have any more questions.

ntuwork commented 3 months ago

Thank you for reaching out and for your interest in the code!

Testing indeed requires generating prediction results which then need to be submitted to the official leaderboard. This is an essential step for evaluation and comparison against other methods. We are unable to perform offline testing primarily because the test set does not come with ground truth values accessible to us. This limitation means that to obtain accurate and comparable results, submission to the leaderboard, which has access to the ground truths, is necessary.

I hope this clarifies your concerns. Please feel free to reach out if you have any more questions.

Thanks a lot for your answer! I noticed that the link of paper is 404notfound, can you recover it ?

XiaolongTang23 commented 3 months ago

Unfortunately, due to my advisor's decision not to upload the paper to arXiv, the only available version is the initial draft on OpenReview. Accessing this draft requires logging in to the OpenReview platform. Without logging in, the website will display an "Error 403" message.

ntuwork commented 3 months ago

Unfortunately, due to my advisor's decision not to upload the paper to arXiv, the only available version is the initial draft on OpenReview. Accessing this draft requires logging in to the OpenReview platform. Without logging in, the website will display an "Error 403" message.

Thanks a lot!

zachytong commented 3 months ago

Unfortunately, due to my advisor's decision not to upload the paper to arXiv, the only available version is the initial draft on OpenReview. Accessing this draft requires logging in to the OpenReview platform. Without logging in, the website will display an "Error 403" message.

Thanks a lot!

Hi, I came across the same problem, and after logging in the page with 403 error wll be refreshed to a homepage, do you solve this problem?