search

rashindrie / DIPA

Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-domain Few-Shot Learning - CVPR 2024
MIT License
10 stars 2 forks source link

readme

Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-domain Few-Shot Learning

Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learning
Rashindrie Perera, Saman Halgamuge

CVPR 2024

arXiv 2403.04492

Pre-trained model checkpoints

We release following pre-trained checkpoints using Masked Image Modelling (MIM) for reproducibility. 1) Pre-trained on eight datasets: MDL checkpoint 2) Pre-trained only on ImageNet-train set: SDL checkpoint

Additionally, the SDL-E checkpoints which were already available and used in our work can be accessed via below links: 1) MIM Pre-trained on ImageNet-full set: SDL-E MIM checkpoint. 2) DINO Pre-trained on ImageNet-full set: SDL-E DINO checkpoint.

Prerequisites

Please download and install Pytorch 1.9.0 and TensorFlow 2.6.0. This code was tested on Python 3.8.6 and CUDA 11.1.1.

pip install -r requirements.txt

Datasets

We utilize the Meta-Dataset for our main results. Instructions for downloading and pre-processing Meta-Dataset can be found here. We provide a dataset class for Meta-Dataset to be used during pre-training under the datasets folder. We also provide the label files created for MIM pre-training on Meta-Dataset here: label_folder.

Pre-training

We mainly follow the hyperparameters provided for pre-training using MIM while additionally following the author's recommendations to set teacher patch temperature to 0.04 instead of the default 0.07 provided in the source code. 


export NCCL_SOCKET_IFNAME="bond0.3027,p1p1.3027"
export NCCL_IB_HCA=mlx5_bond_0,mlx5_0
export NCCL_IB_GID_INDEX=7

export MASTER_ADDR=127.0.0.1
export MASTER_PORT=12345
export WORLD_SIZE=4

CHECKPOINT_DIR='./output_folder' # define path here

python -m torch.distributed.launch --use_env --nproc_per_node $WORLD_SIZE main_pretrain.py  \
  --dataset 'meta-dataset' \
  --data_path ${PATH_TO_META_DATASET_FOLDER} \
  --label_path ${PATH_TO_LABEL_FOLDER} \
  --output_dir ${CHECKPOINT_DIR} \
  --arch vit_small \
  --norm_last_layer false --use_fp16 True \
  --image_size 224 --local_crops_size 96 --patch_size 16 \
  --batch_size_per_gpu 128 \
  --epochs 800 \
  --shared_head true \
  --out_dim 8192 \
  --local_crops_number 10 \
  --global_crops_scale 0.25 1 \
  --local_crops_scale 0.05 0.25 \
  --pred_ratio 0 0.3 \
  --pred_ratio_var 0 0.2 \
  --teacher_temp 0.04 --teacher_patch_temp 0.04 --warmup_teacher_temp_epochs 30 --warmup_epochs 10 \

To pre-train an MDL backbone set --dataset meta-dataset else, to pre-train a SDL backbone set --dataset imagenet. Default choice is meta-dataset.

Meta-Testing

Meta-Dataset

Place/run the below code snippet which is required for using the MetaDataset readers, before running the evaluation scripts.

ulimit -n 50000

export META_DATASET_ROOT='/data/gpfs/projects/punim1193/few-shot-experiments/simple-cnaps/meta-dataset/'
export META_DATASET_ROOT='/data/gpfs/projects/punim1193/public_datasets/meta-dataset/'

export DATASRC='/data/gpfs/projects/punim1193/public_datasets/meta-dataset/data'
export SPLITS='/data/gpfs/projects/punim1193/public_datasets/meta-dataset/splits'
export RECORDS='/data/gpfs/projects/punim1193/public_datasets/meta-dataset/processed_data'

Evaluation:

CUDA_VISIBLE_DEVICES=0 python -u test_extractor.py \
    --pretrained_setting 'MDL' --test_type 'standard' \
    --out_dir ${RESULTS_PATH} --checkpoint_path ${PATH_TO_CHECKPOINT}

Ensure that checkpoint_path points to the pre-trained checkpoint and out_dir points to the results folder in which you need to save evalution results.

To reproduce the N-way-K-shot tasks results presented in main text, set the test_type as standard:

--test_type 'standard'

To reproduce the varying-way-5-shot results in main text, set test_type as 5shot: 

--test_type '5shot'

For running evaluation on an MDL pre-trained checkpoint, set pretrained_setting as MDL. 

--pretrained_setting 'MDL'

Otherwise, use SDL or SDL_E for running evaluation on other settings.

--pretrained_setting 'SDL' or --pretrained_setting 'SDL_E'

Additional analysis on CIFAR-FS and mini-ImageNet

Datasets

We utilize CIFAR-FS, and Mini-ImageNet for additional evaluations. Please refer our Supplementary material for results from the additional evaluations.

tar -xvf cifar_fs.tar cifar_fs/


- Mini-ImageNet can be downloaded [here](https://cseweb.ucsd.edu/~weijian/static/datasets/mini-ImageNet/).

tar -xvf MiniImagenet.tar.gz miniimagenet_224/


We also provide dataset classes for above datasets under the `datasets` folder.

## Meta-Testing

CUDA_VISIBLE_DEVICES=0 python -u test_extractor_others.py \ --n_way ${N_WAY} --k_shot ${K_SHOT} --dataset ${DATASET_NAME} \ --checkpoint_path ${PATH_TO_CHECKPOINT} --data_path ${DATASET_PATH} --out_dir ${RESULTS_PATH} 

Set `${DATASET_NAME}` as `cifar-fs` or as `mini_imaget` for CIFAR-FS and mini-ImageNet datasets, respectively. 
Here, the convention is to evaluate `5-way-5-shot` or `5-way-1-shot`. Define `N_WAY`, and `K_SHOT` according to the specific task you need to evaluate.

For example, for evaluating the 5-way-5-shot setting for cifar-fs dataset:

CUDA_VISIBLE_DEVICES=0 python -u test_extractor_others.py \ --n_way 5 --k_shot 5 --dataset cifar_fs \ --checkpoint_path ${PATH_TO_CHECKPOINT} --data_path ${DATASET_PATH} --out_dir ${RESULTS_PATH} 


## Citation

If you find our project helpful, please consider to cite our paper:

Rashindrie Perera, Saman Halgamuge; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 23794-23804

@InProceedings{Perera_2024_CVPR, author = {Perera, Rashindrie and Halgamuge, Saman}, title = {Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learning}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2024}, pages = {23794-23804} }