The images used in this evaluation are drawn from the CheXpert dataset. Please make sure to download the CheXpert dataset with all images before proceeding.
image-retrieval: This directory hosts files for the image-image retrieval evaluation.candidate.csv: This file contains the paths of 8x200 candidate images, along with original labels in the CheXpert dataset. The paths are relative to the original CheXpert download directory.
query.csv: This file contains expert-selected query images. For each one of the 8 disease categories, 10 query images are provided.
text-retrieval: This directory hosts files for the text-image retrieval evaluation.candidate.csv: This file contains the paths of the same 8x200 candidate images drawn from the CheXpert dataset.
query.csv: This file contains expert-written text queries. For each one of the 8 disease categories, 10 textual queries are provided.
A retrieved image from all the candidates is considered correct only if its label matches the label of the query image or text. For example, for a query image from the Atelectasis category, a retrieved image is only considered a positive retrieval if its label for Atelectasis is 1. The retrieval metrics such as Precision@10 are then calculated based on this criterion.
For details on how the data is collected and used, please refer to this paper.