SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery

SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery (ECCV 2024)
By Sarah Rastegar, Mohammadreza Salehi, Yuki Asano, Hazel Doughty, and Cees Snoek.

Dependencies

pip install -r requirements.txt

kmeans_pytorch Installation

Since our work relies heavily on kmeans_pytorch for cluster assignments, you need to ensure that it is correctly imported to reproduce the results from the paper. You can install kmeans_pytorch directly in the directory by executing the following commands:

cd SelEx
git clone https://github.com/subhadarship/kmeans_pytorch
cd kmeans_pytorch
pip install --editable .

Note: While using scikit-learn's KMeans provides improvements, the results in the paper have been reported using kmeans_pytorch.

Config

Set paths to datasets, pre-trained models and desired log directories in config.py

Set SAVE_DIR (logfile destination) and PYTHON (path to python interpreter) in bash_scripts scripts.

Datasets

We use fine-grained benchmarks in this paper, including:

• The Semantic Shift Benchmark (SSB) and Herbarium19

We also use generic object recognition datasets, including:

• CIFAR-10/100 and ImageNet

Scripts

Train representation: To run the code with the hyperparameters used in the paper, execute the following command:

python contrastive_training.py

This script will automatically train the representations, extract features, and fit the semi-supervised KMeans algorithm. It also provides final evaluations on both the best checkpoint and the final checkpoint.

Dataset Hyperparameter Specifics: If you're working with the CUB dataset, set the unsupervised_smoothing parameter to 1.0, for other fine-grained datasets (Scars, Pets, Aircraft) to 0.5 and for generic datasets to 0.1.

Evaluation

In the Final Reports section at the end, please note that only the evaluations reported for:

Reports for the best checkpoint:
Reports for the last checkpoint:

are the evaluations performed at test time to evaluate the checkpoints. Additionally, please note that Train ACC Unlabelled_v2 is the metric reported by our work and prior studies.

:clipboard: Citation

@inproceedings{RastegarECCV2024,
title = {SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery},
author = {Sarah Rastegar and Mohammadreza Salehi and Yuki M Asano and Hazel Doughty and Cees G M Snoek},
year = {2024},
booktitle = {European Conference on Computer Vision},
}

The codebase is mainly built on the repo of https://github.com/sgvaze/generalized-category-discovery.