SYSU-HCP-at-ImageCLEF-VQA-Med-2021

This repository is the official implementation of paper SYSU-HCP at VQA-Med 2021: A Data-centric Model with Efficient Training Methodology for Medical Visual Question Answering.

We won first prize in ImageCLEF 2021 VQA-Med - VQA.

Main requirements

• torch == 1.4.0
• torchvision == 0.5.0
• tensorboardX == 2.1
• Python 3

The environment can be created following this:

# First, create a virtual environment
conda create -n your_env_name python=3.6
conda activate your_env_name

# Second, install the required packages from requirements.txt
conda install pip
pip install -r requirements.txt

Models

The ResNeSt needs to be installed before using.

Datasets

The training set, validation set and test set can be downloaded from the Google Drive directly and should be put in the data folder.

Training

You can train the model from scratch. The command and corresponding parameters are as follows:

python train.py [-input_size <int>] [-batch_size <int>] [-backbone <model_name>] [-train_set <Med_LT_train or train>] [-gpu <int>] [-world_size <int>] [-port <int>] [--mixup] [--label_smooth] [--superloss] [-dryrun] [-resume_path <checkpoint.pth>] [-pretrain <checkpoint.pth>]

Evaluation

You can evaluate the trained model and save the results in a csv file.

python validation.py -model_path <checkpoint.pth> -csv_path <path_to_save_evaluation_results> [-input_size <int>] [-backbone <model>] [-gpu <int>]

Pretrained models for VQA-Med 2021

We provide the pretrained models for VQA-Med 2021 in Google Drive and Baidu Cloud (code:i1nn).

After downloading the models, you can get the result by the following command:

python inference_ensemble_once.py

Citing this repository

If you find this code useful in your work, please consider citing us:

@inproceedings{gong2021sysu-hcp,
  author    = {Haifan Gong and
               Ricong Huang and
               Guanqi Chen and
               Guanbin Li},
  title     = {SYSU-HCP at VQA-Med 2021: A Data-centric Model with Efficient Training Methodology for Medical Visual Question Answering},
  booktitle = {CLEF 2021 -- Conference and Labs of the Evaluation Forum, September 21--24, 2021, Bucharest, Romania},
  series    = {CEUR Workshop Proceedings},
  year      = {2021},
}