# Vocabulary-free Image Classification and Semantic Segmentation [Alessandro Conti](https://scholar.google.com/citations?user=EPImyCcAAAAJ), [Enrico Fini](https://scholar.google.com/citations?user=OQMtSKIAAAAJ), [Massimiliano Mancini](https://scholar.google.com/citations?user=bqTPA8kAAAAJ), [Paolo Rota](https://scholar.google.com/citations?user=K1goGQ4AAAAJ), [Yiming Wang](https://scholar.google.com/citations?user=KBZ3zrEAAAAJ), [Elisa Ricci](https://scholar.google.com/citations?user=xf1T870AAAAJ)

Setup

Install dependencies

# clone project
git clone https://github.com/altndrr/vicss
cd vicss

# install requirements
# it will create a .venv folder in the project root
# and install all the dependencies using flit
make install

# activate virtual environment
source .venv/bin/activate

Setup environment variables

# copy .env.example to .env
cp .env.example .env

# edit .env file
vim .env

Usage

The two entry points are train.py and eval.py. Calling them without any argument will use the default configuration.

# train model
python src/train.py

# test model
python src/eval.py

Configuration

The full list of parameters can be found under configs, but the most important ones are:

• data: dataset to use, default to caltech101.
• experiment: experiment to run, default to baseline/clip.
• logger: logger to use, default to null.

Parameters can be overwritten by passing them as command line arguments. You can additionally override any parameter from the config file by using the ++ prefix.

# train model on ucf101 dataset
python src/train.py data=ucf101 experiment=baseline/clip

# train model on ucf101 dataset with RN50 backbone
python src/train.py data=ucf101 experiment=baseline/clip model=clip ++model.model_name=RN50

Note that since all our approaches are training-free, there is virtually no difference between train.py and eval.py. However, we still keep them separate for clarity.

Development

Install pre-commit hooks

# install pre-commit hooks
pre-commit install

Run tests

# run fast tests
make test

# run all tests
make test-full

Format code

# run linters
make format

Clean repository

# remove autogenerated files
make clean

# remove logs
make clean-logs

Acknowledgements

We gratefully acknowledge taap studio for designing the logo of this project and ashleve/lightning-hydra-template for the template used to build this repository.