search

houzhijian / GroundNLQ

The champion solution for Ego4D Natural Language Queries Challenge in CVPR 2023
MIT License
14 stars 1 forks source link

Fail to reproduce the "Training-From-Scratch" performance #1

Open Becomebright opened 12 months ago

Becomebright commented 12 months ago

Thanks for your impressive work.

I try to reproduce the "Training-From-Scratch" performance using: bash tools/train_ego4d_twogpu.sh configs/ego4d_nlq_v2_internvideo_1e-4.yaml scratch_2gpu 0,1. The evaluation result is about 14-15 R@1, IoU=0.3 on the validation set, significantly lower than the reported number. Could you please give me some advice on reproducing the "Training-From-Scratch" performance?

First, there's a typo in README. I think the "Training-From-Scratch" command should be bash tools/train_ego4d_twogpu.sh configs/ego4d_nlq_v2_internvideo_1e-4.yaml scratch_2gpu 0,1.

Moreover, trunc_thresh and crop_ratio in ego4d_nlq_v2_internvideo_1e-4.yaml have to be deleted.

Further, the learning_rate in ego4d_nlq_v2_internvideo_1e-4.yaml is 5e-5, contradicting with the config name.

Could you please check these potential mistakes and provide a reproducible config?

Yisen-Feng commented 9 months ago

Thanks for your impressive work.

I try to reproduce the "Training-From-Scratch" performance using: bash tools/train_ego4d_twogpu.sh configs/ego4d_nlq_v2_internvideo_1e-4.yaml scratch_2gpu 0,1. The evaluation result is about 14-15 R@1, IoU=0.3 on the validation set, significantly lower than the reported number. Could you please give me some advice on reproducing the "Training-From-Scratch" performance?

First, there's a typo in README. I think the "Training-From-Scratch" command should be bash tools/train_ego4d_twogpu.sh configs/ego4d_nlq_v2_internvideo_1e-4.yaml scratch_2gpu 0,1.

Moreover, trunc_thresh and crop_ratio in ego4d_nlq_v2_internvideo_1e-4.yaml have to be deleted.

Further, the learning_rate in ego4d_nlq_v2_internvideo_1e-4.yaml is 5e-5, contradicting with the config name.

Could you please check these potential mistakes and provide a reproducible config?

I have the same problem. Have you solved it? Just to add, the number of epochs is not consistent with what is in the paper.

Becomebright commented 9 months ago

No, I didn't try any further.