Does the spatial API only work with audiosprites?

[roschler]

I'm having a truly painful time trying to get the Spatial audio working. I've combed through the 3D demonstration many times over, matching my code to its, down to the pannerAttr settings used for the speaker that plays music through the speaker. I've compared my HowlerGlobal position settings in conjunction with my virtual world object position settings and compare the values to the those between the Howler 3D listener/speaker position settings.

If I don't make a set position call to the Howl() object for my virtual world sound, then I hear the sound just fine. But as soon as I make that call I can only hear it if I crank my speakers up to full blast and then it is very faint. Worse, as I rotate or move about the world, the sound's volume or perceived location does not change. In fact, due to a delay one time between the play start call and the subsequent position and orientation calls I make after that, I heard the sound blast out of my speakers for about a 1/10th of a second before being clamped down to being nearly inaudible.

At this point, the main difference between my code and the 3D code is that I am initializing the Howl() object from a source URL and not an audio sprites file. Does the spatial audio API only work with audio sprites? If not, what else could I be doing wrong that I could investigate? Note, I have not set html5 to TRUE since I did see that other issue where you stated that the spatial API does not work with HTML5 (streaming?) audio.

[roschler]

UPDATE: Too soon to call this a bug but I am seeing something truly odd in the code in howler.core.js. It really does seem that the library expects the sound to be an audio sprite. Here is the line of code where I hear the sound get clamped down to nearly inaudible in howler.spatial.js:

    if (!sound._paused) {
      sound._parent.pause(sound._id, true).play(sound._id, true);
    }

So when that module sets up the panner, it pauses the sound and then immediately resumes it after setting up the panner and some other things. However, when I trace through the play() in that statement I see the code below get fires:

      // End the sound instantly if seek is at the end.
      if (seek >= stop) {
        self._ended(sound);
        return;
      }

So the code thinks the sound is already ended and returns immediately instead of resuming. I believe the reason I hear the sound incredibly softly is because it is looped, and the sound is being muted quickly each time by this problem, but I can't say anything for certain right now. I say it seems to be sprite related because the code above that self._ended call seems to making decisions base on an audio sprite being in use. So I am wondering again, does the spatial library work with sounds that are source from a URL and not an audio sprites file?

[goldfire]

Did you find a solution to this?

[roschler]

@goldfire Yes. It's a fairly complicated situation so I've been waiting until I have firmed up my knowledge before answering. I'll try now:.

It was a combination of misunderstandings and having some difficult problems to solve, especially when it came to translating back and forth between the ThreeJS coordinate system and the Howler coordinate system:

1) I was creating my own global howler object with "new HowlerGlobal()". Because of this the audio context was NULL and that killed the use of the panner nodes (another concept had to learn before I could solve my problems).

2) I had to firmly grasp the concept of the sound IDs. I had thought, from my use of Howler way back before I tried to use the spatial plug-in, that they were just an optional item when making Howler calls. Instead they are critical. To me, it was confusing that calling functions that hung directly off the Howl object itself still required the sound ID.

3) The deep difficulties of translating the ThreeJS coordinate system, which uses 3D position and Eulerian orientation angles for world objects, to the simpler paradigm shown in the Howler 3D demo. That demo uses a simple angle in radians rotating around a Y-axis while passing the sin and cosine of the angle as the X and Z coordinates to the Howler orientation setting function, with the Y-axis value set to 0. Things got better when I created a function with this code in it:

    let vecToReceiveWorldDir = new THREE.Vector3();

    // Convert a ThreeJS object's world direction to an angle around the Y-axis,
    //  as required by our use of Howler.js.
    g_ThreeJsCamera.getWorldDirection(vecToReceiveWorldDir);
    const theta = Math.atan2(vecToReceiveWorldDir.x, vecToReceiveWorldDir.z);

    return new THREE.Vector3(Math.cos(theta), 0, Math.sin(theta));

Although for some reason, I have to swap the X and Z values to get proper alignment.

However it was issue #2 that really tripped me up. To use the spatial plugin you really have to understand what the sound IDs are and even more importantly, when to use them. One of the aspects of those IDs that can really tripped me up is that you don't get the ID until you call the "play()" function, since it returns the sound ID, as shown in the "speaker:" function in the Howler 3D sample.

Perhaps there is a better way, but that left me having to do my position and orientation setting in the "sound playing" event handler I set up for my Howl object, since you don't have the sound ID until the sound starts playing. I think it would be helpful if the "play()" function was modified to accept a callback that is called when the "play()" code has obtained the sound ID.

Another alternative is if the Howl constructor code would look to the initialization JSON block passed to the Howl() constructor and if there are properties for orientation or position, assign the Web audio object orientation and position for you at that time. I looked in the code for the latest version of Howler core and I didn't see anywhere it looks for properties with those names. My apologies in advance if I missed them.

I think one of the biggest problems for someone who has difficulties implementing the spatial API, is the decoupling of the Howl object's "_pos" and "_orientation" properties and the actual Web audio API object that also has those properties. It is during tracing I discovered how my lack of understanding of the sound IDs was the root cause of many of my problems.

During tracing, I saw that those two properties in the Howl object are still null after you set the orientation and position for a Howl object. That's when I discovered this code in the core Howler module:

const soundObj = self._soundById(arySoundIds[0]);

And that is where you find the actual position and orientation values, not in the Howl object. Because of this, I added these two functions to my copy of howler.spatialjs:

 /**
   * Get the position for a Howl object using its ID.
   *
   * @return {*} - The current position of the sound.
   */
  Howl.prototype.getPositionBySoundId = function(id) {
    const errPrefix = '(Howl.prototype.getPositionBySoundId) ';

    const self = this;

    if (misc_shared_lib.isEmptySafeString(id))
      throw new Error(errPrefix + `The id parameter is empty.`);

    const arySoundIds = self._getSoundIds(id);

    if (arySoundIds.length <= 0)
      return null; // Sound not found using the given ID.

    const soundObj = self._soundById(arySoundIds[0]);

    if (!soundObj)
      throw new Error(errPrefix + `The sound object returned for sound ID '${id}' is NULL.`);

    return soundObj._pos;
  }

  /**
   * Get the orientation for a Howl object using its ID.
   *
   * @return {*} - The current orientation of the sound.
   */
  Howl.prototype.getOrientationBySoundId = function(id) {
    const errPrefix = '(Howl.prototype.getOrientationBySoundId) ';

    const self = this;

    if (misc_shared_lib.isEmptySafeString(id))
      throw new Error(errPrefix + `The id parameter is empty.`);

    const arySoundIds = self._getSoundIds(id);

    if (arySoundIds.length <= 0)
      return null; // Sound not found using the given ID.

    const soundObj = self._soundById(arySoundIds[0]);

    if (!soundObj)
      throw new Error(errPrefix + `The sound object returned for sound ID '${id}' is NULL.`);

    return soundObj._orientation;
  }

These functions allowed me to get the actual position and orientation values so I could inspect the Howl objects values and compare them against the ThreeJS graphic object's position and orientation value, so I could debug and fix the translations between my ThreeJS world coordinates and the the Howl theta-angle values.

The Howler library is a powerful library and I see you went to great lengths to "recover" from various browser audio environment deficiencies. The library intelligently falls back to various available contexts given the user's browser platform and system configuration when advanced audio features are not available in the user's environment.

Unfortunately, those fallback measures cover up contexts because they "fail silently" thereby squelching problems the developer might need to see because the real problem isn't a deficiency in the browser environment, but a mistake the developer made in configuring Howler, like when I created my own copy of Howler global instead of using the singleton object the Howler core module sets up for you. This could easily be solved by simply adding a console.warn() statement every time the Howler core module falls back to a lesser context, with the warning message explaining what was not available and what context was being switched to in order to continue.

Thanks for asking and thank you for creating howler.