Harmonic Networks

Repository implements harmonic blocks presented in following papers:

• Harmonic Networks for Image Classification
• Harmonic Networks with Limited Training Samples
• Harmonic Convolutional Networks based on Discrete Cosine Transform

Convolutional neural networks (CNNs) learn filters in order to capture local correlation patterns in feature space. In this paper we propose to revert to learning combinations of preset spectral filters by switching to CNNs with harmonic blocks. We rely on the use of the Discrete Cosine Transform (DCT) filters which have excellent energy compaction properties and are widely used for image compression. The proposed harmonic blocks rely on DCT-modeling and replace conventional convolutional layers to produce partially or fully harmonic versions of new or existing CNN architectures. We demonstrate how the harmonic networks can be efficiently compressed in a straightforward manner by truncating high-frequency information in harmonic blocks which is possible due to the redundancies in the spectral domain. We report extensive experimental validation demonstrating the benefits of the introduction of harmonic blocks into state-of-the-art CNN models in image classification, segmentation and edge detection applications.

Requirements

All code compatible with Python 3.5+ and PyTorch 1.0+ Install the required packages by running:

pip install -r requirements.txt

Installation

First build the package prior to running any code:

python setup.py develop

Repository implements experiments on several datasets:

• Image classification on CIFAR10/100
• Image classification on STL10
• Image classification on ImageNet
• Object detection on MS COCO and PASCAL VOC
• Boundary prediction on BSDS500

Citation

@article{Ulicny22,
  title = {Harmonic convolutional networks based on discrete cosine transform},
  journal = {Pattern Recognition},
  volume = {129},
  pages = {108707},
  year = {2022},
  issn = {0031-3203},
  doi = {https://doi.org/10.1016/j.patcog.2022.108707},
  url = {https://www.sciencedirect.com/science/article/pii/S0031320322001881},
  author = {Matej Ulicny and Vladimir A. Krylov and Rozenn Dahyot},
}

@inproceedings{Ulicny19b,
  title = {Harmonic Networks for Image Classification},
  author = {Ulicny, Matej and Krylov, Vladimir A and Dahyot, Rozenn},
  booktitle={Proceedings of the British Machine Vision Conference},
  year = {2019},
  month={Sep.}
}

@inproceedings{Ulicny19,
  author = {Ulicny, Matej and Krylov, Vladimir A and Dahyot, Rozenn},
  booktitle={27th European Signal Processing Conference (EUSIPCO)},
  title = {Harmonic Networks with Limited Training Samples},
  doi={10.23919/EUSIPCO.2019.8902831},
  ISSN={2219-5491},
  year={2019}, 
  month={Sep.},
}

License

This project is licensed under the BSD License - see the LICENSE file for details.