Kashu7100
commented
1 year ago It seems some of the input features are missing. Could you double-check the data and path used in your VLCAP/src/rtransformer/recursive_caption_dataset.py?
Closed IQraQasim10 closed 1 year ago
Kashu7100
commented
1 year ago It seems some of the input features are missing. Could you double-check the data and path used in your VLCAP/src/rtransformer/recursive_caption_dataset.py?
IQraQasim10
commented
1 year ago Oh. it solved the issue of missing features and did work in retrieving the env, lang and sent features. Could you please let me know where would I have to make these changes for evaluation part? Also, I would like to mention, I added
import nltk nltk.download('punkt')
in src/train.py and the training worked perfect.
Would you recommend working with wandb?
Kashu7100
commented
1 year ago Can you change the data path in VLCAP/src/rtransformer/recursive_caption_dataset.py if you put your data somewhere else?
I think wandb is a useful tool to monitor the training.
IQraQasim10
commented
1 year ago My training results are pretty much similar to what you have mentioned in Table 5 in VLCAP paper. As I trained my model on anet dataset and feature= true. I am expecting to evaluate using _bash scripts/translate_greedy.sh anet_re_init_2022_11_30_16_2643 val command. But I ran into the following problem.
(pytorch) iqa000@vs-c1:~/Workspace/VLCAPmain$ bash scripts/translate_greedy.sh anet_re_init_2022_11_30_16_26_43 val train_opt Namespace(attention_probs_dropout_prob=0.1, batch_size=4, beam_size=2, cuda=True, data_dir='/home/iqa000/Workspace/VLCAP-main/densevid_eval/anet_data', debug=False, dset_name='anet', ema_decay=0.9999, eval_tool_dir='./densevid_eval', exp_id='init', freeze_voc=False, grad_clip=1, hidden_dropout_prob=0.1, hidden_size=768, initializer_range=0.02, intermediate_size=768, label_smoothing=0.1, layer_norm_eps=1e-12, log='results/anet_re_init_2022_11_30_16_26_43/model', lr=0.0001, lr_warmup_proportion=0.1, max_es_cnt=10, max_n_sen=6, max_t_len=22, max_v_len=100, memory_dropout_prob=0.1, mtrans=False, n_best=1, n_epoch=50, n_memory_cells=1, no_cuda=False, no_pin_memory=False, num_attention_heads=12, num_hidden_layers=2, num_workers=8, pin_memory=True, recurrent=True, res_dir='results/anet_re_init_2022_11_30_16_26_43', res_root_dir='results', resume_path=None, run_id='3t09b6h8', save_mode='best', save_model='results/anet_re_init_2022_11_30_16_26_43/model', seed=2019, share_wd_cls_weight=False, type_vocab_size=2, untied=False, use_beam=False, use_env=True, use_lang=True, v_duration_file='/home/iqa000/Workspace/VLCAP-main/video_feature/anet_duration_frame.csv', val_batch_size=12, video_feature_dir='/home/iqa000/Workspace/VLCAP-main/video_feature/anet_trainval', video_feature_size=2048, voc_path='/home/iqa000/Workspace/VLCAP-main/cache/anet_vocab_clip.pt', vocab_size=10655, word2idx_path='/home/iqa000/Workspace/VLCAP-main/cache/anet_word2idx.json', word_vec_size=512, xl=False, xl_grad=False) Start evaluating val Traceback (most recent call last): File "src/translate.py", line 236, in <module> main() File "src/translate.py", line 186, in main eval_data_loader = get_data_loader(opt, eval_mode=eval_mode) File "src/translate.py", line 116, in get_data_loader recurrent=opt.recurrent, untied=opt.untied or opt.mtrans) File "/home/iqa000/Workspace/VLCAPmain/src/rtransformer/recursive_caption_dataset.py", line 87, in __init__ self._load_duration() File "/home/iqa000/Workspace/VLCAPmain/src/rtransformer/recursive_caption_dataset.py", line 165, in _load_duration with open(self.duration_file, "r") as f: FileNotFoundError: [Errno 2] No such file or directory: '/home/iqa000/Workspace/VLCAP-main/video_feature/anet_duration_frame.csv'
When I navigate to anet_duration_frame.csv I see that the file is here but empty. Does it have to anything with shutting down they system after training? Could you please figure this out...!
Kashu7100
commented
1 year ago When I navigate to anet_duration_frame.csv I see that the file is here but empty.
The duration file should contain the duration info. Can you download the corresponding file again from the main branch of this repo?
Also, please double check your path (/home/iqa000/Workspace/VLCAP-main/video_feature/anet_duration_frame.csv).
IQraQasim10
commented
1 year ago YES. rechecking with the setup.sh and resolving the path solved the issue. The Evaluation is on its way ...
2022-12-02 12:51:35,615: Mode val
2022-12-02 12:51:35,616: Loading data from /home/iqa000/Workspace/VLCAP-main/densevid_eval/anet_data/anet_entities_val_1.json
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2460/2460 [00:00<00:00, 1284929.99it/s]
2022-12-02 12:51:35,669: Loading complete! 2460 examples
0%| | 0/2460 [00:00<?, ?it/s]missing
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2460/2460 [00:00<00:00, 53009.26it/s]
Missing 1 features (clips/sentences) from 1 videos
Missing {'j73Wh1olDsA'}
[Info] Trained model state loaded.
- (Translate): 20%|██████████████████████████████████████▊
What is your recommendation on yc2 dataset and training plus evaluation.
IQraQasim10
commented
1 year ago I would like to share the results anet-test
{
"Bleu_1": 0.5353381506518596,
"Bleu_2": 0.33391063758245343,
"Bleu_3": 0.21240864474730795,
"Bleu_4": 0.13808044790562626,
"METEOR": 0.1728796180211062,
"ROUGE_L": 0.36290273965969605,
"CIDEr": 0.30131769467803965,
"gt_stat": {
"avg_sen_len": 14.806818181818182,
"num_sen": 8712,
"vocab_size": 5510
},
"submission": {
"avg_sen_len": 11.630165289256198,
"num_sen": 8712,
"vocab_size": 1453
},
"num_evaluated": 2456,
"num_gt": 2457,
"num_pred": 2456,
"re1": 0.39192528249105507,
"re2": 0.17046172781410823,
"re3": 0.10296342311520865,
"re4": 0.06931700901990746
}anet-val
{
"Bleu_1": 0.534599739771031,
"Bleu_2": 0.33526446720433706,
"Bleu_3": 0.21478895575947762,
"Bleu_4": 0.14042879116199797,
"METEOR": 0.17454953193732303,
"ROUGE_L": 0.3650001728174786,
"CIDEr": 0.3180997324647702,
"gt_stat": {
"avg_sen_len": 14.912959160392425,
"num_sen": 8766,
"vocab_size": 5547
},
"submission": {
"avg_sen_len": 11.611339265343371,
"num_sen": 8766,
"vocab_size": 1431
},
"num_evaluated": 2459,
"num_gt": 2460,
"num_pred": 2459,
"re1": 0.3953617894613977,
"re2": 0.17605731717542453,
"re3": 0.10709377472488246,
"re4": 0.07456230463937506
}Thank you so much for this wonderful code
Kashu7100
commented
1 year ago For the yc2 feature extraction, please refer to our recently published code at https://github.com/UARK-AICV/VLTinT.
Hi, after running bash scripts/train.sh anet true generates there is a long list of
But then it displays the following error.
File "/home/iqa000/Workspace/VLCAP-main/src/train.py", line 684, in
main()
File "/home/iqa000/Workspace/VLCAP-main/src/train.py", line 630, in main
train_loader = DataLoader(train_dataset, collate_fn=collate_fn,
File "/home/iqa000/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 344, in init
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File "/home/iqa000/.local/lib/python3.10/site-packages/torch/utils/data/sampler.py", line 107, in init
raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0****
I am following read.me like a holy thing and there is data also in the data folder. (env, lang, sens) After running evaluation command _bash scripts/translate_greedy.sh anet_re_init_2022_11_22_17_1143 test I also get similar type of error. Is there some issue with dataloader?
Also, I noticed that after running _bash scripts/translate_greedy.sh anet_re_init_2022_11_22_17_1143 test results/anetre*/greedy_pred_val.json json file is note created.