AppliedAcousticsChalmers / sound_field_analysis-py

Analyze, visualize, and process sound field data recorded by spherical microphone arrays.
https://appliedacousticschalmers.github.io/sound_field_analysis-py/
MIT License
91 stars 16 forks source link

SN3D normalized spherical harmonics #20

Open abhayap opened 3 years ago

abhayap commented 3 years ago

Is it possible to add support for SN3D normalized spherical harmonics? SN3D is used as a standard in many systems now.

HaHeho commented 3 years ago

We recently added the option to decompose and render real spherical harmonics, see Exp4.

I am not super familiar with the different conventions, so I would have to check whether the one we deployed already is SN3D. https://github.com/AppliedAcousticsChalmers/sound_field_analysis-py/blob/838c57e24e780bcc752ad393d41cabcbc7e891df/sound_field_analysis/sph.py#L480-L488

If not, it should be fairly straight forward to incorporate a different convention. Would you be able to assist with that?

HaHeho commented 3 years ago

We recently added the option to decompose and render real spherical harmonics, see Exp4.

This version is not yet released as the PyPI package, in case that was the source for installing the package. We will update it in the upcoming days. Otherwise, you can install the local code from the repository.

I am not super familiar with the different conventions, so I would have to check whether the one we deployed already is SN3D.

After checking, what we are currently using is ACN ordering and the N3D normalization?

Also our coefficients include the Condon-Shortley phase (according to the scipy). I'm guessing SN3D might use factor 1 instead of (-1)^m?

@JensAhrens could you verify?

If not, it should be fairly straight forward to incorporate a different convention. Would you be able to assist with that?

abhayap commented 3 years ago

Thanks for the quick reply! You can find a definition of the SN3D normalization here: http://www.matthiaskronlachner.com/wp-content/uploads/2013/01/ICSA2014_KronlachnerZotter_AmbisonicTransformations.pdf

It shouldn't be too hard to add support for that. There's also a way to transform the N3D to SN3D. I don't remember the exact formula.

abhayap commented 3 years ago

You can find more information and a way to convert from N3D to SN3D here: https://en.wikipedia.org/wiki/Ambisonic_data_exchange_formats

HaHeho commented 3 years ago

You can find more information and a way to convert from N3D to SN3D here: https://en.wikipedia.org/wiki/Ambisonic_data_exchange_formats

Yes, that is what I looked at as well.

One remark: to our method. We usually use time domain -> frequency domain -> SH domain -> complex multiplication -> summation -> time domain , while Ambisonics considers the exchange of time domain signals, so time domain -> SH domain -> convolution -> summation (whereby the convolution is probably also implemented as a fast convolution in frequency domain).

There shouldn't be a reason to not be able to do time domain -> SH domain -> frequency domain -> complex multiplication -> summation -> time domain with the tools implemented here, but we have not tested this AFAIK. However, this should not be difficult to verify.

The actually used convention only matters when exchanging coefficients with other pipelines for rendering. If I may ask, what use case or way of rendering where you considering?

abhayap commented 3 years ago

One remark: to our method. We usually use time domain -> frequency domain -> SH domain -> complex multiplication -> summation -> time domain ,

The final time domain HOA signal needs to have been produced by SN3D normalized spherical harmonics to be properly decoded in various software that accept HOA inputs like: https://facebookincubator.github.io/facebook-360-spatial-workstation/KB/CreatingVideosSpatialAudioFacebook360.html?highlight=sn3d

chris-hld commented 3 years ago

Converting from N3D to SN3D is just an order weighting: https://github.com/chris-hld/spaudiopy/blob/ae71d0cfec1cfd587e71eb085c501bcb4c86bf49/spaudiopy/sph.py#L246

Feel free to copy if needed!

chris-hld commented 3 years ago

I am not super familiar with the different conventions, so I would have to check whether the one we deployed already is SN3D.

https://github.com/AppliedAcousticsChalmers/sound_field_analysis-py/blob/838c57e24e780bcc752ad393d41cabcbc7e891df/sound_field_analysis/sph.py#L480-L488

That should be N3D.

With _np.float_power(-1.0, m) you are removing the Condon-Shortley phase, however, it looks like only for half of them? I assume you should check that it is removed in both, m greater and less than 0. Here is a better reference, with corrections, as the SH plot in the paper above has a flipped sign:

https://www.researchgate.net/profile/Franz_Zotter/publication/266602800_AMBIX_-A_SUGGESTED_AMBISONICS_FORMAT/links/58160c0208aeb720f68800fb/AMBIX-A-SUGGESTED-AMBISONICS-FORMAT.pdf

HaHeho commented 3 years ago

Thanks Chris for these references. 👍

I also noticed that our versions leads to a slightly different result compared to your implementation. However, I did not have the time investigate this closer. Will do so at some point. https://github.com/chris-hld/spaudiopy/blob/ae71d0cfec1cfd587e71eb085c501bcb4c86bf49/spaudiopy/sph.py#L98-L117

chris-hld commented 3 years ago

Might be the phase. If there is something else happening than a flip around the z axis, please let me know!