minzwon / sota-music-tagging-models

MIT License
403 stars 65 forks source link

State-of-the-art Music Tagging Models

License: MIT

PyTorch implementation of state-of-the-art music tagging models :notes:

Demo and Docker image on Replicate

Reference

Evaluation of CNN-based Automatic Music Tagging Models, SMC 2020 [arxiv]

-- Minz Won, Andres Ferraro, Dmitry Bogdanov, and Xavier Serra

TL;DR

Available Models

Requirements

conda create -n YOUR_ENV_NAME python=3.7
conda activate YOUR_ENV_NAME
pip install -r requirements.txt

Preprocessing

STFT will be done on-the-fly. You only need to read and resample audio files into .npy files.

cd preprocessing/

python -u mtat_read.py run YOUR_DATA_PATH

Training

cd training/

python -u main.py --data_path YOUR_DATA_PATH

Options

'--num_workers', type=int, default=0
'--dataset', type=str, default='mtat', choices=['mtat', 'msd', 'jamendo']
'--model_type', type=str, default='fcn',
                choices=['fcn', 'musicnn', 'crnn', 'sample', 'se', 'short', 'short_res', 'attention', 'hcnn']
'--n_epochs', type=int, default=200
'--batch_size', type=int, default=16
'--lr', type=float, default=1e-4
'--use_tensorboard', type=int, default=1
'--model_save_path', type=str, default='./../models'
'--model_load_path', type=str, default='.'
'--data_path', type=str, default='./data'
'--log_step', type=int, default=20

Evaluation

cd training/

python -u eval.py --data_path YOUR_DATA_PATH

Options

'--num_workers', type=int, default=0
'--dataset', type=str, default='mtat', choices=['mtat', 'msd', 'jamendo']
'--model_type', type=str, default='fcn',
                choices=['fcn', 'musicnn', 'crnn', 'sample', 'se', 'short', 'short_res', 'attention', 'hcnn']
'--batch_size', type=int, default=16
'--model_load_path', type=str, default='.'
'--data_path', type=str, default='./data'

Performance Comparison

Performances of SOTA models

Performances with perturbed inputs ## Citation ``` @inproceedings{won2020eval, title={Evaluation of CNN-based automatic music tagging models}, author={Won, Minz and Ferraro, Andres and Bogdanov, Dmitry and Serra, Xavier}, booktitle={Proc. of 17th Sound and Music Computing}, year={2020} } ``` ## License ``` MIT License Copyright (c) 2020 Music Technology Group, Universitat Pompeu Fabra. Code developed by Minz Won. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ``` ## Upcoming Models Available upon request. minz.won@upf.edu