Binaural audio on positional audio

[lucasmr]

So, I've tested positional audio on CS:GO and Rocket League, and it seems that even after ticking the box "Headphones" on the audio setup does not correlate to binaural audio. IE: someone speaking on my left generates audio only on the left side of the headphone, instead of having a sound on both sides, but delayed ~0.7ms and attenuated on the right.

Is it working as intended, or is the feature missing?

[hacst]

Mumble's positional audio isn't true binaural audio. What we use is a simple stereo panning scheme. As such what you are experiencing is the expected behavior (though having no sound at all on the right side would require quite precise positioning). While this approach is pretty simplistic it works quite well in a lot of situations and is easy to implement and cheap w.r.t to runtime.

Of course doing something more sophisticated like a real HRTF instead would produce much more realistic results (including being able to resolve front/back as well as up/down properly). There has been some research on adding this to Mumble in the past (in the form of a research prototype and paper in Mumble 1.1.X times) but iirc the HRTF library used for that had some licensing issues and noone stepped up to further pursue the issue.

TL;DR: You could call it a missing feature ;)

[lucasmr]

Nice, thanks for the clarification.

Maybe add this as a feature request?

[davidebeatrici]

@lucasmr Yes, I think it's a good idea, so we can talk about it and how to implement it, if it's possible :wink: @hacst What about OpenAL? Has it already been considered?

[hacst]

@davidebeatrici According to Wikipedia OpenAL has a proprietary license since version 1.1 (and before that it was LGPL) which pretty much rules out using it in Mumble. There seems to be an alternative implementation https://github.com/kcat/openal-soft (did you mean that one?). However that is also LGPL which is problematic. http://www.bitoutsidethebox.com/shabda/hrtf-info/ has a description how it can be used. It also references http://recherche.ircam.fr/equipes/salles/listen/index.html as a source for HRTFs. Unfortunately I couldn't find any licensing information on those files (which doesn't necessarily mean they are restrictively licensed, we would have to find out). Licensing of the HRTFs was also a problem in the experimental implementation.

I haven't done any kind of real survey about what other implementations are out there though.

I'll see if I can dig up the old paper and code (and whether I am allowed to publish it...). If someone wants to take a stab at adding this functionality to Mumble we would definitely be willing to support such an effort.

[davidebeatrici]

@hacst I checked the licenses here, I didn't know that LGPL would be a problem; I also gave a look at the one you linked. There are some open source alternatives, such as libsoundio, but I can't find any info regarding binaural audio. What is the old code based on?

[hacst]

Well. Qt itself is LGPL so it isn't out of the question I guess. The code was based on Mumble 1.2.2 and was likely an original implementation. I haven't found my copy yet though :( I found the paper . It's called "Integrating a HRTF-based sound synthesis system into Mumble" but it's IEEE licensed and behind a paywall (see here). I don't think I can redistribute my copy but you probably get access to it easily enough ;)

[davidebeatrici]

Done :wink: I see that there are a lot of interesting things in this paper :smile:

[Hiradur]

I already proposed to use OpenAL Soft in #1933. The LGPL license of OAL Soft shouldn't be a showstopper and since it's the de facto standard for 3D audio on Linux it's well tested and in wide adoption. In the issue ticket linked above I also give some reasons why I think it's a good idea to use OAL Spft. The information given at http://www.bitoutsidethebox.com/shabda/hrtf-info/ is outdated btw. OAL Soft now turns on HRTF automatically when headphones are detected (can be disabled in the config) and there is also an API for games to toggle HRTF.

[DaveKap]

There are bots for Discord and Mumble which allow broadcasting of audio from sources like Soundcloud and YouTube. With Discord, stereo broadcasting works so if the source has proper left and right broadcasting, you'll hear it from the bot. Unfortunately, Mumble's bots can't broadcast this stereo yet because this feature is missing! So consider this comment a "please help me keep my friends from wanting to use Discord just because Mumble is missing this feature" vote up for adding stereo broadcasting to Mumble. ;)

[Hiradur]

@DaveKap This issue is about binaural audio which is something different. You should create a new ticket for stereo transmissions if none exists already.

[foresto]

The sample audio in the PR that closed this issue sounds to me not much different from panning. I guess the inter-channel delay it adds is what was originally called for here, but given that this discussion has evolved to include HRTF, would it make sense to either re-open this issue or create a new one for the more advanced functionality?

It would be impressive if mumble could do positional audio as well as I remember from Overwatch, where I could hear when someone was (for example) above and behind my left shoulder so reliably that I could swivel around to aim at them in a single motion.

[Krzmbrzl]

@foresto I think a new, separate issue would be best

[foresto]

Ah: I see now that #1933 covers this (although its title doesn't make it clear). I don't think we need another report for it. :)

[Krzmbrzl]

Yes indeed :+1: