This repository implements a DINO (Distillation of Self-Supervised Learning) model, focusing on self-supervised learning techniques for computer vision. It's based on Vision Transformers (ViTs) and includes custom neural network architectures and data augmentation strategies.
Trainer.py
: Manages model training, including setup and execution.train.py
: Main script to configure and start the training process.eval.py
: Main script to evaluate the model on kNN accuracy.utils.py
: Provides utility functions and classes for model training.models/DINO.py
: Contains the DINO model implementation.models/DINO_head.py
: Contains the DINO head implementation.models/DINO_loss.py
: Contains the DINO loss implementation. (TODO)configs/
: YAML configuration files for setting up models and datasets and Pydantic models for model validation.To train the DINO model:
configs
folder.train.py
with the desired configuration file.To evaluate the DINO model:
training_output
eval.py
with the desired configuration from the CLI commandFor more information on the underlying concepts and methodologies, refer to the original paper: Emerging Properties in Self-Supervised Vision Transformers.