How is the test conducted?

[ntuwork]

Thank you for making your code public!

I have a question about how to do testing for interaction dataset. For the interaction dataset, is it necessary to go to the official leaderboard to upload for multi-agent testing and get the results? Is it feasible to test offline? I have observed that many methods do not perform offline testing and do not provide code for offline testing. I would like to know what are the reasons for this.

Thanks a lot.

[XiaolongTang23]

Thank you for reaching out and for your interest in the code!

Testing indeed requires generating prediction results which then need to be submitted to the official leaderboard. This is an essential step for evaluation and comparison against other methods. We are unable to perform offline testing primarily because the test set does not come with ground truth values accessible to us. This limitation means that to obtain accurate and comparable results, submission to the leaderboard, which has access to the ground truths, is necessary.

I hope this clarifies your concerns. Please feel free to reach out if you have any more questions.

[ntuwork]

Thank you for reaching out and for your interest in the code!

Testing indeed requires generating prediction results which then need to be submitted to the official leaderboard. This is an essential step for evaluation and comparison against other methods. We are unable to perform offline testing primarily because the test set does not come with ground truth values accessible to us. This limitation means that to obtain accurate and comparable results, submission to the leaderboard, which has access to the ground truths, is necessary.

I hope this clarifies your concerns. Please feel free to reach out if you have any more questions.

Thanks a lot for your answer! I noticed that the link of paper is 404notfound, can you recover it ?

[XiaolongTang23]

Unfortunately, due to my advisor's decision not to upload the paper to arXiv, the only available version is the initial draft on OpenReview. Accessing this draft requires logging in to the OpenReview platform. Without logging in, the website will display an "Error 403" message.

[ntuwork]

Unfortunately, due to my advisor's decision not to upload the paper to arXiv, the only available version is the initial draft on OpenReview. Accessing this draft requires logging in to the OpenReview platform. Without logging in, the website will display an "Error 403" message.

Thanks a lot!

[zachytong]

Unfortunately, due to my advisor's decision not to upload the paper to arXiv, the only available version is the initial draft on OpenReview. Accessing this draft requires logging in to the OpenReview platform. Without logging in, the website will display an "Error 403" message.

Thanks a lot!

Hi, I came across the same problem, and after logging in the page with 403 error wll be refreshed to a homepage, do you solve this problem?

[zhaozhihua97]

感谢您的联系以及对代码的关注！

测试确实需要生成预测结果，然后需要将其提交给官方排行榜。这是评估和与其他方法进行比较的重要步骤。我们无法进行离线测试，主要是因为测试集没有我们可以访问的地面真值。这一限制意味着，要获得准确且可比较的结果，必须提交给可以访问地面真值的排行榜。

希望这能解答您的疑虑。如果您还有其他问题，请随时联系我们。

你好，请问Argoverse也需要提交到官方验证吗？如果不是，我没有看到代码里包括如何计算指标。