search

facebookresearch / EgoVLPv2

Code release for "EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone" [ICCV, 2023]
MIT License
83 stars 11 forks source link

Interpretation of validation results #11

Open Dongzhikang opened 5 months ago

Dongzhikang commented 5 months ago

Dear author, I really appreciate your work. I was able to reproduce validation on EgoMCQ test. After validation, I found there are two lines of output in info.log

2024-02-10 16:52:08,386 - trainer - INFO - EgoClip epoch -1, Intra-video, Acc: 90.9; EgoClip epoch -1, Inter-video, Acc: 60.7;
2024-02-10 16:52:09,525 - trainer - INFO - EgoClip epoch -1, Intra-video, Acc: 90.5; EgoClip epoch -1, Inter-video, Acc: 59.6;

I have two questions:

  1. Are there typos about "Intra-video" and "Inter-video"? Because this is not consistent with the results in your paper.
  2. It seems like the first line of output is the results in paper, how to interpret the second line of output?
ShramanPramanick commented 5 months ago

Hi @Dongzhikang,

I'm glad you could resolve the errors and reproduce the results. Please find the answers below:

  1. Good catch. In the current version, I mistakenly swapped the keywords 'Inter-video' and 'Intra-video' while logging the results. I will update this soon.
  2. The first printed line, EgoClip epoch -1, Intra-video, Acc: 90.9; EgoClip epoch -1, Inter-video, Acc: 60.7, is the results achieved by the ensemble of dual-encoder and fusion-encoder. The second printed line, EgoClip epoch -1, Intra-video, Acc: 90.5; EgoClip epoch -1, Inter-video, Acc: 59.6, is the result obtained by the fusion-encoder. We have reported these results in the last row of Table 8.
ShramanPramanick commented 4 months ago

The 'Inter-video' and 'Intra-video' keywords are now updated to be consistent. Please let me know if you have any further questions, and feel free to close the issue if it is resolved.