Spectrum Correction: Acoustic Scene Classification with Mismatched Recording Devices

This repository contains source code of the experiments presented in

"Spectrum Correction: Acoustic Scene Classification with Mismatched Recording Devices".

DOI: 10.21437/Interspeech.2020-3088

Slides from Interspeech 2020 can be found in ./Interspeech 2020 slides.mp4

Table of Contents

1. Description
   • Selected Results
2. Training
   • Installation
   • Quick Start
   • Preprocessing
   • Experiments
3. License
4. Citation

Description

Machine learning algorithms, when trained on audio recordings from a limited set of devices, may not generalize well to samples recorded using other devices with different frequency responses.

In this work, a relatively straightforward method is introduced to address this problem. Two variants of the approach are presented. First requires aligned examples from multiple devices, the second approach alleviates this requirement.

This method works for both time and frequency domain representations of audio recordings. Further, a relation to standardization and Cepstral Mean Subtraction is analysed. The proposed approach becomes effective even when very few examples are provided.

This method was developed during the Detection and Classification of Acoustic Scenes and Events (DCASE) 2019 challenge and won the 1st place in the scenario with mismatched recording devices with the accuracy of 75%.

Selected Results

Accuracy for devices A, B and C for models trained with and without Spectrum Correction and using other methods on the TAU Urban Acoustic Scenes 2019 Mobile dataset. See the paper for the full table.

Training

Installation

Installation requires CUDA 10.0 and the corresponding CuDNN. The dataset will be downloaded automatically (default location is ./data/dcase/TAU-urban-acoustic-scenes-2019-mobile-development).

git clone <repository-url> spectrum-correction
cd spectrum-correction
virtualenv -p python3 venv
. venv/bin/activate
pip install -r requirements.txt

Quick Start

Download and preprocess the dataset, then train the model:

. venv/bin/activate
./prepare-dcase.py 0 data/dcase.h5
./train.py --reproducible --mixup-exp --mixup 0.4 data/dcase.h5 baseline

The raw dataset should get downloaded and extracted automatically, when ./prepare-dcase.py is executed for the first time.

Launch TensorBoard to view the results:

. venv/bin/activate
tensorboard --logdir logs

Preprocessing

Dataset for training is prepared using the ./prepare-dcase.py script, which downloads and prepares the data. Options can be view using

./prepare-dcase.py --help

Dataset with default settings (0 removes the limit on the number of examples used to fit the correction):

./prepare-dcase.py 0 data/dcase.h5

Dataset with default settings and maximum of 32 examples for SC:

./prepare-dcase.py 32 data/dcase-32-pairs.h5

Dataset made using the aligned variant of SC and with device b as the reference device:

./prepare-dcase.py --aligned --reference b 0 data/dcase-b-reference.h5

Dataset created using Spectrum Correction with implementation based on Finite Input Response (FIR) filter:

./prepare-dcase.py --aligned --fir 0 data/dcase-fir.h5

Dataset using a randomized cross validation split (seed 16) and reusing examples from developement set:

./prepare-dcase.py --reuse --split 16 0 data/dcase-validation-16.h5

Experiments

Training is performed using the ./train.py script. Options can be viewed using

./train.py --help

All experiments used the following invocation with different dataset and experiment name.

./train.py --reproducible --mixup-exp --mixup 0.4 <dataset> <experiment-name>

For example:

./train.py --reproducible --mixup-exp --mixup 0.4 data/dcase.h5 baseline

License

This source code is relased under AGPL v3 license. See the LICENSE file.

Citation

If you find this repository or the paper useful, please cite them as:

@inproceedings{Kosmider2020,
  author={Michał Kośmider},
  title={{Spectrum Correction: Acoustic Scene Classification with Mismatched Recording Devices}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={4641--4645},
  doi={10.21437/Interspeech.2020-3088},
  url={http://dx.doi.org/10.21437/Interspeech.2020-3088}
}