How to obtain the ADE/FDE/MR result of test set?

[WenchuanSUN1]

I would like to express my appreciation for your work on the HiVT project. I have been using HiVT for my research, and it has been a valuable tool for my experiments. I am currently working on evaluating the performance of HiVT on the test set of Argoverse. However, I cannot correct load test set data. I always meet the error:

HiVT2/models/hivt.py", line 129, in validation_step
    l2_norm = (torch.norm(y_hat[:, :, :, : 2] - data.y, p=2, dim=-1) * reg_mask).sum(dim=-1)  # [F, N]
TypeError: unsupported operand type(s) for -: 'Tensor' and 'NoneType'

I would greatly appreciate it if you could provide some guidance or assistance.

[nanybeih]

@WenchuanSUN1 I meet the same error. Do u have solved this problem? I would greatly appreciate it if you could provide some guidance. Thank u very much.

[yyjs666]

The test set of Argoverse contains only the information for the first 2s, so data.y does not exist（NoneType）

[WenchuanSUN1]

The test set of Argoverse contains only the information for the first 2s, so data.y does not exist（NoneType）

I appreciate your feedback. However, it has come to my attention that a number of widely-recognized methods, as detailed in their respective publications, have conducted experiments utilizing the Argoverse test dataset. Hence, it is reasonable to infer that standard experiments at the data level should be feasible, as exemplified by TNT, LaneConv, and JEAN.

[Uestc-Young]

The test set of Argoverse contains only the information for the first 2s, so data.y does not exist（NoneType）

I appreciate your feedback. However, it has come to my attention that a number of widely-recognized methods, as detailed in their respective publications, have conducted experiments utilizing the Argoverse test dataset. Hence, it is reasonable to infer that standard experiments at the data level should be feasible, as exemplified by TNT, LaneConv, and JEAN.

Maybe you can put your output to Argoversev1 leaderboard, and you can get the metrics(ade, fde, mr, etc.) through the official website. There is also a function in argoverse-api to help you submit to the leaderboard, I suppose the function should be:

from argoverse.evaluation.competition_util import generate_forecasting_h5