About your map in LAM Task

[lemon-prog123]

Hi! Recently I used the checkpoint named 'lam-gaze360-best.pth' that you provide in google drive. It got 0.7990 map in val set just as the paper said. But when I use it to get a result on test set.,it only get 0.7197 map and 0.859 acc. I wonder that if there is something wrong with map results in your paper? The map in test set should be 0.78 in your paper.

[lemon-prog123]

By the way, I get the result from test set on evalai.

[zcxu-eric]

Hello, the arxiv paper is out of date, these are our final results.

[lemon-prog123]

Hello, the arxiv paper is out of date, these are our final results.

Ok I see. But I wonder that whether the map 72.11 produced by your released model on colab? Or you had trained a new model.

[lemon-prog123]

The question is due to the result I get from your release model is only 0.7197 on EvalAI. However, one possible reason is that the csv file may cause some accuracy loss when submit on EvalAI.

[zcxu-eric]

Hello, the results reported in the paper is based on the original dataset (with gt bboxes, tracklets labels). But we can't release the original labels due to the conflicts with other challenges in Ego4d. We provide the identical checkpoint but the cropped dataset. The baseline results on EvalAI is produced by this checkpoint on the cropped data, that should be the baseline for this challenge.