How to calculate OKVQA image features

[ifmaq1]

Can you please tell that where can I get the annotation files of OKVQA dataset that are used to calculate image features only.

[prdwb]

Thank you for the question. Image features are extracted from the images using a pretrained fast rcnn model: https://github.com/huggingface/transformers/blob/master/examples/research_projects/lxmert/extracting_data.py We have extracted features that can be downloaded via the link in the README file. Thanks.

[ifmaq1]

@prdwb thank you for your response. Your mentioned link has this line

python extracting_data.py -i <img_dir> -o <dataset_file>.datasets <batch_size>

could you please tell me what did you write in the above command line to give appropriate paths.

Also, okvqa dataset contains approx 15k images. Did you calculate the features of only those images? or you calculated the features of whole coco dataset with coco-annotations and later filtered the images. If so, how did you do that. As I am unable to find the annotations and images(only for 15k images that okvqa dataset use)

[prdwb]

I picked out okvqa images from all coco images and put them in a folder. I then ran the command on this folder. This can be done by something like this:

import os
import json
from shutil import copy2
from tqdm import tqdm

images_path = 'the path that contains all coco images'
okvqa_path = 'OpenEnded_mscoco_{}_questions.json'
data_sub_types = ['train2014', 'val2014']

output_path = 'the output folder'

for data_sub_type in data_sub_types:
    with open(okvqa_path.format(data_sub_type)) as fin:
        data = json.load(fin)

    questions = data['questions']
    for question in tqdm(questions):
        image_id = question['image_id']
        image_filename = f'COCO_{data_sub_type}_{str(image_id).zfill(12)}.jpg'
        image_path = os.path.join(images_path.format(data_sub_type), image_filename)
        copy2(image_path, output_path)

Thanks.

[ifmaq1]

Thanks a lot for this information.