Hybrid Structural Model for HRTF individualization

About

This repository includes the code for a hybrid structural HRTF model combining measured, synthesised, and selected components [1]. In particular, its three components are:

• A generic head-and-torso component, taken from the "pinna-less" KEMAR set included in the Viking HRTF dataset v2 [2] with ITD removed (measured component);
• A fully customized pinna component, built using features related to the shape of the user’s pinnae through deep learning [1,3] (synthesized component);
• The best-match ITD from an available HRTF dataset [4] obtained by regression on anthropometric parameters of the head and torso [5] (selected component).

The model, implemented in MATLAB/Python, directly outputs a SOFA file.

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

To run this project you'll need a copy of Python 3.x (3.7 or later recommended) and MATLAB. If you want to use the generated SOFA file with Steam Audio, it is recommended to use MATLAB 2016. See this issue for more details.

Make sure to download and install the following MATLAB toolboxes:

• SOFA API (1.1.x): https://sourceforge.net/projects/sofacoustics/files
• Auditory Modeling Toolbox (latest): http://amtoolbox.sourceforge.net

Furthermore, you'll need a copy of the HUTUBS [4] dataset, which can be found here (file HRIRs.zip).

Finally, it is recommended to install the necessary Python dependencies within a virtualenv or using conda. In the latter case, make sure to install pip within your Conda environment first.

Installation

1. Clone the repo

   git clone git@github.com:miccio-dk/itsadive_hybrid_structural_model.git

2. Install Python packages

   cd ./itsadive_hybrid_structural_model/python
   pip install -r requirements.txt

Usage

In order to generate an individualized HRTF set from user's data, follow these steps.

Anthropometric data collection

• Place an absolute reference (such as a ruler) behind the ear and take a photo of the pinna from ~30 cm afar.
• Extract the pinna contours
  • Using an image editing program such as GIMP, use the measuring tool to derive the quantity W, equivalent to 9.5 cm in pixels.
  • Use the selection tool to mark a square of side W centered at the entrance of the ear canal and crop to selection.
  • Trace the pinna contours manually or with the help of MATLAB's edge() function; the areas of interest are the contour of the concha (cymba and cavum), the ear canal, the tragus, and the inner and outer edges of the helix.
  • Scale the image down to 256x256 pixels and store at PNG.
• Measure the head width from above the ears, the head depth between the forehead and the outermost part of the back, and the shoulder circumference using a soft measuring tape around the chest

PRTF generation

From within the python/ directory, run the following command:

python ear_to_prtf.py configs/edges_median.json /path/to/pinna_contours.png --nfft 512 --output_path /path/to/prtf.mat

For more info regarding further arguments and options:

python ear_to_prtf.py --help

HRTF generation

Open MATLAB and call generateHrtfSet() with the following arguments:

• head_width, head_depth, and shoulder_circumference measured earlier, in centimeters
• the path to the PRTF generated using the Python script
• the output path to the generated SOFA file

License

Distributed under the MIT License. See LICENSE for more information.

Citation

If you use this code in a scientific publication, please reference the following works [1,2]:

@inproceedings{micciniHybridApproachStructural2021,
    title = {A hybrid approach to structural modeling of individualized {HRTFs}},
    booktitle = {2021 {IEEE} {Conference} on {Virtual} {Reality} and {3D} {User} {Interfaces} {Abstracts} and {Workshops} ({VRW} 2021)},
    author = {Miccini, R. and Spagnol, S.},
    month = mar,
    year = {2021}
}

@misc{spagnolVikingHRTFDataset2020,
    title = {The {Viking} {HRTF} dataset v2},
    url = {https://zenodo.org/record/4160401},
    publisher = {Zenodo},
    author = {Spagnol, Simone and Miccini, Riccardo and Unnthorsson, Runar},
    month = oct,
    year = {2020},
    doi = {10.5281/zenodo.4160401},
    note = {type: dataset},
}

Contact

Simone Spagnol - @itsadive - ssp (@) create.aau.dk

Project Link: https://itsadive.create.aau.dk

References

• [1] R. Miccini and S. Spagnol (2021). A hybrid approach to structural modeling of individualized HRTFs. In: Proceedings of the 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Workshops (VRW 2021), Lisbon, Portugal, March 2021.

• [2] S. Spagnol, R. Miccini, and R. Unnthórsson (2020). The Viking HRTF dataset v2. DOI: 10.5281/zenodo.4160401

• [3] R. Miccini and S. Spagnol (2020). HRTF individualization using deep learning. In: Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Workshops (VRW 2020), pages 390-395, Atlanta, GA, USA, March 2020.

• [4] F. Brinkmann, M. Dinakaran, R. Pelzer, J.J. Wohlgemuth, F. Seipl, and Stefan Weinzierl (2019). The HUTUBS HRTF database. DOI: 10.14279/depositonce-8487

• [5] S. Spagnol (2020). HRTF selection by anthropometric regression for improving horizontal localization accuracy. IEEE Signal Processing Letters 27, pages 590-594, April 2020.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 797850.