search

cshizhe / VLN-DUET

Official implementation of Think Global, Act Local: Dual-scale GraphTransformer for Vision-and-Language Navigation (CVPR'22 Oral).
106 stars 8 forks source link

The herustic to choose pretrain weight? #13

Closed MarSaKi closed 1 year ago

MarSaKi commented 1 year ago

Hi, thanks for your great work! I found the performance of DUET on REVERIE dataset highly depends on the initial pretrained weight. For example, ckpt20000 and ckpt40000 result in different finetuning performance, though they have similar performance on pretraining tasks (MLM, SAP, OG). Do you have any heuristic to choose the best pretrained weight (which iteration) to initialize DUET?

Best,

cshizhe commented 1 year ago

Hi, I run an online testing with the pretained ckpts and select the checkpoint with highest navigation performance.

HonsenZhang commented 1 year ago

Hi, I run an online testing with the pretained ckpts and select the checkpoint with highest navigation performance.

@cshizhe Hi, could you please tell me what exactly dose the "navigation performance" mean? Is it the global/local/fused accuracy of SAP? or something else? Thanks