junyangwang0410 / Knight

SotA text-only image/video method (IJCAI 2023)
11 stars 0 forks source link

请教一下数据集中的coco_test.txt是什么内容?我从官网下载的coco2014 image captioning 数据集中没有看到这个。 #3

Open kk-dark opened 4 months ago

kk-dark commented 4 months ago
    json_path = "./data/COCO/captions_val2014.json"
    json_labels = json.load(open(json_path,'r'))
    annotations = json_labels["annotations"]
    images = json_labels["images"]
    images_path = "./data/COCO/image/"

    image_dict = dict()
    for image in images:
        image_dict[image["file_name"]] = image["id"]

    with open("./data/COCO/coco_test.txt") as image_names_data:
        image_names = image_names_data.readlines()

    image_features = []
    for image_info in image_names:
        image_file = image_info.split('\n')[0]
        image_id = image_dict[image_file]
        image_path = images_path + image_file
        ori_image = Image.open(image_path)
        image = preprocess(ori_image).unsqueeze(0).to(device)
        image_feature = clip_model.encode_image(image)
        image_features.append(image_feature)

    image_features = torch.cat(image_features)
    torch.save(image_features, "./feature/COCO/image_features.pkl")

因为不懂这个coco_test.txt文件,这段代码没有看明白,如果是读取图片的话,应该只需要拼接file_name与image_folder_name吧。

junyangwang0410 commented 4 months ago

coco_test.txt是目前测试COCO的Image Captioning指标的一个主流测试集数据划分,许多工作都在此基础上进行测试,我们为了公平比较,因此follow了这个设置。这里是为了将图片转为对应的id,便于查找。

kk-dark commented 4 months ago

coco_test.txt是目前测试COCO的Image Captioning指标的一个主流测试集数据划分,许多工作都在此基础上进行测试,我们为了公平比较,因此follow了这个设置。这里是为了将图片转为对应的id,便于查找。

明白了,万分感谢您的回复!