Can you release all the model parameters?

[jhliu17]

Hi author, I am interested in your work and I have tried to reproduce this project in YouCook2 validation dataset. But the result let me down... I don't know why I cannot realize the result reported in your paper.

I follow the setting what you write in the readme. The model parameters what I used is below: --max_sentence_len: 20 --d_mode: 1024 --d_hidden: 2048 --n_heads: 8 --in_emb_dropout: 0.1 --attn_dropout: 0.2 --vis_emb_dropout', default=0.1 --cap_dropout', default=0.2 --image_feat_size', default=3072 --n_layers', default=2 --train_sample', default=20 --sample_prob', default=0

--slide_window_size', default=480 --slide_window_stride', default=20 --sampling_sec', default=0.5 --kernel_list', default=[1, 2, 3, 4, 5, 7, 9, 11, 15, 21, 29, 41, 57, 71, 111, 161, 211, 251] --pos_thresh', default=0.7 --neg_thresh', default=0.3 --stride_factor', default=50

--max_epochs', default=20 --batch_size', default=16 --valid_batch_size', default=16 --cls_weight', default=1.0 --reg_weight', default=10 --sent_weight', default=0.25 --scst_weight', default=0.0 --mask_weight', default=1.0 --gated_mask', action='store_true'

--optim',default='sgd' --learning_rate', default=0.1 --alpha', default=0.95 --beta', default=0.999 --epsilon', default=1e-8 --loss_alpha_r', default=2 --patience_epoch', default=1 --reduce_factor', default=0.5 --grad_norm', default=1

The result what I got is: Proposal recall area: 18.823 BLEU@3: 2.749 BLEU@4: 0.6991 Meteor: 8.4377

And I found that the validation loss reduces to 1.2+ and It cannot be optimized for further reduction. Does it seem that the model still in the under-fitting state? The one reason I can figure out is that the model parameters I set are not the same as what you use. Hope to get your answer soon. Thanks!

[LuoweiZhou]

Hi @jhliu17, thanks for your interest in our work. We use the same configuration as in the code we provided, as in README and here. We also provided the pre-trained model on youcook2 as mentioned in the README. It yields 1.16 in Bleu4 and 8.73 in METEOR. Since you had changes in the hyper-parameters, e.g., batch_size, you might want to tweak the learning_rate a little bit.

[jhliu17]

Thanks for @LuoweiZhou kindly answer~! I find the reason is the validation loss cannot truly reflect the metric result, and it seems that I need to finetune the model carefully.

[LuoweiZhou]

@jhliu17 great to know that! Closing the issue for now.