search

cshizhe / VLN-DUET

Official implementation of Think Global, Act Local: Dual-scale GraphTransformer for Vision-and-Language Navigation (CVPR'22 Oral).
114 stars 7 forks source link

Val-unseen SR&SPL in pretrain and finetune #12

Closed Hoyyyaard closed 1 year ago

Hoyyyaard commented 2 years ago

Hello，When I apply the official code for pretrain ，the val-unseen sr is about 48 of the best ckpt； Then I apply this ckpt to the finetune code ， it easily reach about 65 of val-unseen sr after epoch 20000，but the val-unseen sr does not raise obviously at the rest of the finetune process; We cannot reproduce the val-unseen eval standrd in the paper and we do not know which part went wrong.Can u please inform us some information?

cshizhe commented 2 years ago

How do you select the best checkpoint? It seems to be a similar issue as this.

Hoyyyaard commented 2 years ago

I run an online testing with the pretained ckpts and select the checkpoint with highest navigation performance too; Can you please inform us the sr and spl of your best pretrained ckpts? Thanks!

MarSaKi commented 2 years ago

I run an online testing with the pretained ckpts and select the checkpoint with highest navigation performance too; Can you please inform us the sr and spl of your best pretrained ckpts? Thanks!

I guess it is informed in the Table.3 of the mainpaper.

Hoyyyaard commented 2 years ago

Sorry， I forgot to emphasize that this data was ran on R2R