SEM-PCYC

PyTorch | Arxiv

PyTorch implementation of our SEM-PCYC model for zero-shot sketch-based image retrieval:
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval
Anjan Dutta, Zeynep Akata
CVPR, 2019

Demo Video

Live Demo

Retrieval Results

Sketchy

TU-Berlin

Prerequisites

• Linux (tested on Ubuntu 16.04)
• NVIDIA GPU + CUDA CuDNN
• 7z

  sudo apt-get install p7zip-full

  Getting Started

Installation

• Clone this repository

  git clone https://github.com/AnjanDutta/sem-pcyc.git
  cd sem-pcyc

• Install the requirements (not checked)

  pip3 install -r requirements.txt

• Update config.ini (see example)

  [<host>]
  path_dataset = <where all the datasets will be downloaded>
  path_aux = <where all the auxiliary folders and files will be saved>

  Download datasets

• Sketchy
• TU-Berlin

  bash download_datasets.sh

  Download pretrained models

• Sketchy
  • sketch
  • image
  • hieremb-jcn + word2vec-google-news
• TU-Berlin
  • sketch
  • image
  • hieremb-path + word2vec-google-news

    bash download_models.sh

    Test

    Sketchy

    python3 src/test.py --dataset Sketchy_extended --dim-out 64 --semantic-models hieremb-jcn word2vec-google-news

    TU-Berlin

    python3 src/test.py --dataset TU-Berlin --dim-out 64 --semantic-models hieremb-path word2vec-google-news

    Train

    Sketchy

    python3 src/train.py --dataset Sketchy_extended --dim-out 64 --semantic-models word2vec-google-news --epochs 1000 --early-stop 200 --lr 0.0001

    TU-Berlin

    python3 src/train.py --dataset TU-Berlin --dim-out 64 --semantic-models word2vec-google-news --epochs 1000 --early-stop 200 --lr 0.0001

    Citation

    @inproceedings{Dutta2019SEMPCYC,
    author = {Anjan Dutta and Zeynep Akata},
    title = {Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval},
    booktitle = {CVPR},
    year = {2019}
    }

Author

• Anjan Dutta (@AnjanDutta)