overte-org / overte

Overte open source virtual worlds platform.
https://overte.org/
Other
128 stars 47 forks source link

Some (stereo?) audio files don't spatialize #929

Open HifiExperiments opened 2 months ago

HifiExperiments commented 2 months ago

if you are wearing headphones, sounds will spatialize, i.e. a sound to your right will sound louder in your right ear than your left. but this only works for some sounds.

this one spatializes: http://silverfish-freestuff.s3.amazonaws.com/TestStuff/SoundEntities/dtmf2.mp3

but these two don't: https://files.freemusicarchive.org/storage-freemusicarchive-org/music/no_curator/Scott_Holmes/Inspiring__Upbeat_Music/Scott_Holmes_-_04_-_Upbeat_Party.mp3 https://brainstormer.s3.us-west-2.amazonaws.com/Sounds/Balloon+Popping-SoundBible.com-1247261379.wav

this affects both sound entities and the old audio injectors:

{
  let sound = SoundCache.getSound(<sound URL>);
  let injector = Audio.playSound(sound, {
    position: MyAvatar.position,
    volume: 1.0,
    loop: true,
    pitch: 1.0
  });
}

this is probably related to this block of code in AudioClient.cpp:

                if (options.ambisonic) {
...
                    // spatialize into mixBuffer
                    injector->getLocalFOA().render(_localScratchBuffer, mixBuffer, HRTF_DATASET_INDEX,
                                                   qw, qx, qy, qz, gain, AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                } else if (options.stereo) {
...
                    // direct mix into mixBuffer
                    injector->getLocalHRTF().mixStereo(_localScratchBuffer, mixBuffer, gain,
                                                       AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                } else {  // injector is mono
...
                        // spatialize into mixBuffer
                        injector->getLocalHRTF().render(_localScratchBuffer, mixBuffer, HRTF_DATASET_INDEX,
                                                        azimuth, distance, gain, AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                    } else {
                        // direct mix into mixBuffer
                        injector->getLocalHRTF().mixMono(_localScratchBuffer, mixBuffer, gain,
                                                         AudioConstants::NETWORK_FRAME_SAMPLES_PER_CHANNEL);
                    }
                }

if the comments are to be believed, ambisonic and positional mono sounds spatialize, but stereo sounds only ever mix directly.

1st-BrainStormer commented 2 months ago

I would agree. Stereo could be rendered as 2 Mono Channels so the azimuth, distance and gain would have to be computed for both channels. Right now Mono plays from the center of the entity. Stereo would be offset to the left and right +/- some value from the center. AudioHRTF.h and AudioHRTF.cpp would have to be updated so support stereo Rendering