search

microsoft / TAP

TAP: Text-Aware Pre-training for Text-VQA and Text-Caption, CVPR 2021 (Oral)
MIT License
70 stars 11 forks source link

No targets for training #13

Closed abhinavkcs11 closed 1 year ago

abhinavkcs11 commented 2 years ago

When I train the VQA model I get the warning "Sample list has not field 'targets', are you sure that your ImDB has labels? you may have wanted to run with --evalai_inference 1"

I executed the same command as mentioned: python -m torch.distributed.launch --nproc_per_node 4 tools/run.py --tasks vqa --datasets m4c_textvqa --model m4c_split --seed 13 --config configs/vqa/m4c_textvqa/tap_refine.yml --save_dir save/m4c_split_refine_test --resume_file save/pretrained/textvqa_tap_base_pretrain.ckpt training_parameters.distributed True

Can you provide additional details on this and how to train the model with the targets? And can you point out where the targets and the predictions are getting compared to compute loss?

zyang-ur commented 2 years ago

Hi @abhinavkcs11 ,

The loss function is here: https://github.com/microsoft/TAP/blob/main/pythia/modules/losses.py.

Maybe we could trace from the "sample_list" in the loss functions, back to the data loader (e.g., https://github.com/microsoft/TAP/blob/352891f93c75ac5d6b9ba141bbe831477dcdd807/pythia/datasets/vqa/m4c_textvqa/dataset.py#L296) and see which part is not working as expected? Thank you.