Machine learning challenges for hearing aid processing

We are organising a series of machine learning challenges to enhance hearing-aid signal processing and to better predict how people perceive speech-in-noise (Clarity) and speech-in-music (Cadenza). For further details of the Clarity Project visit the Clarity project website, and for details of our latest Clarity challenges visit our challenge documentation site. You can contact the Clarity Team by email at claritychallengecontact@gmail.com. For further details of the Cadenza Project visit the Cadenza project website, and to find out about the latest Cadenza challenges join the Cadenza Challenge Group.

In this repository, you will find code to support all Clarity and Cadenza Challenges, including baselines, toolkits, and systems from participants. We encourage you to make your system/model open source and contribute to this repository.

Current Events

• The 2nd Cadenza Challenge is now open :fire::fire:
  • Visit the cadenza website for more details.
  • Join the Cadenza Challenge Group to keep up-to-date on developments.
• The 3rd Clarity Enhancement Challenge is now open. :fire::fire:
  • Visit the challenge website for more details.
  • Join the Clarity Challenge Group to keep up-to-date on developments.
• The 2nd Clarity Prediction Challenge (CPC2) is now closed.
• The 4th Clarity Workshop will be held as a satellite event of Interspeech 2023. For details visit the workshop website.

Installation

PyPI

Clarity is available on the Python Package Index (PyPI) to install create and/or activate a virtual environment and then use pip to install.

conda create --name clarity python=3.8
conda activate clarity

pip install pyclarity

GitHub Cloning

# First clone the repo
git clone https://github.com/claritychallenge/clarity.git
cd clarity

# Second create & activate environment with conda, see https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html
conda create --name clarity python=3.8
conda activate clarity

# Last install with pip
pip install -e .

GitHub pip install

Alternatively pip allows you to install packages from GitHub sources directly. The following will install the current main branch.

pip install -e git+https://github.com/claritychallenge/clarity.git@main

Challenges

Current challenge

• The 2nd Cadenza Challege
• The 3rd Clarity Enhancement Challenge

Previous challenges

• The ICASSP 2024 Cadenza Challenge
• The 1st Cadenza Challenge (CAD1)
• The 2nd Clarity Prediction Challenge (CPC2)
• The ICASSP 2023 Clarity Enhancement Challenge
• The 2nd Clarity Enhancement Challenge (CEC2)
• The 1st Clarity Prediction Challenge (CPC1)
• The 1st Clarity Enhancement Challenge (CEC1)

Available tools

We provide also a number of tools in this repository:

• Hearing loss simulation
  • Cambridge MSBG hearing loss simulator: descriptions can be found in the CEC1 description; an usage example can be found in the CEC1 baseline evaluation script evaluate.py.
• Objective intelligibility measurement
  • Modified binaural STOI (MBSTOI): a Python implementation of MBSTOI. It is jointly used with the MSBG hearing loss model in the CEC1 baseline. The official matlab implementation can be found here: http://ah-andersen.net/code/
  • Hearing-aid speech perception index (HASPI): a Python implementation of HASPI Version 2, and the better-ear HASPI for binaural speech signals. For official matlab implementation, request here: https://www.colorado.edu/lab/hearlab/resources
  • Hearing-aid speech quality index (HASQI): a Python implementation of HASQI Version 2, and the better-ear HASQI for binaural speech signals.
  • Hearing-aid audio quality index (HAAQI): a Python implementation of HAAQI.
• Hearing aid enhancement
  • Cambridge hearing aid fitting (CAMFIT): a Python implementation of CAMFIT, translated from the HörTech Open Master Hearing Aid (OpenMHA); the CAMFIT is used together with OpenMHA enhancement as the CEC1 baseline, see enhance.py.
  • NAL-R hearing aid fitting: a Python implementation of NAL-R prescription fitting. It is used as the CEC2 baseline, see enhance.py.

In addition, differentiable approximation to some tools are provided:

• [x] Differentiable MSBG hearing loss model. See also the BUT implementation: https://github.com/BUTSpeechFIT/torch_msbg_mbstoi
• [ ] Differentiable HASPI (coming)

Open-source systems

• CPC1:
  • Exploiting Hidden Representations from a DNN-based Speech Recogniser for Speech Intelligibility Prediction in Hearing-impaired Listeners
  • Unsupervised Uncertainty Measures of Automatic Speech Recognition for Non-intrusive Speech Intelligibility Prediction
• CEC1:
  • A Two-Stage End-to-End System for Speech-in-Noise Hearing Aid Processing