VTPACK

This repo is an official implementation for "Dynamic Grained Encoder for Vision Transformers" (NeurIPS2021) on PyTorch framework.

Installation

Requirements

• Python >= 3.6
• PyTorch >= 1.8 and torchvision
• timm:
  • pip install timm
• GCC >= 4.9

Build from source

• git clone https://github.com/StevenGrove/vtpack
• cd vtpack
• python setup.py build develop

Prepare data

Download and extract ImageNet train and val images from http://image-net.org/. The directory structure is the standard layout for the torchvision datasets.ImageFolder, and the training and validation data is expected to be in the train/ folder and val folder respectively:

/path/to/imagenet/
  train/
    class1/
      img1.jpeg
    class2/
      img2.jpeg
  val/
    class1/
      img3.jpeg
    class/2
      img4.jpeg

Usage

Training

# Running training procedure with specific GPU number
./tools/run_dist_launch.sh <GPU_NUM> <path_to_config> [optional arguments]

# Please refer to main.py for more optional arguments

Inference

# Running inference procedure with specific GPU number and model path
./tools/run_dist_launch.sh <GPU_NUM> <path_to_config> --eval --resume <model_path> [optional arguments]

# Please refer to main.py for more optional arguments

Image Classification on ImageNet val set

More models are comming soon.

Citation

Please cite the paper in your publications if it helps your research.

@inproceedings{song2021dynamic,
    title={Dynamic Grained Encoder for Vision Transformers},
    author={Song, Lin and Zhang, Songyang and Liu, Songtao and Li, Zeming and He, Xuming and Sun, Hongbin and Sun, Jian and Zheng, Nanning},
    booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
    year={2021}
}

Please cite this project in your publications if it helps your research.

@misc{vtpack,
    author = {Song, Lin},
    title = {VTPACK},
    howpublished = {\url{https://github.com/StevenGrove/vtpack}},
    year ={2021}
}