Add support for audio and audio-params

[vincentfretin]

Probably just use the sound aframe component.

[vincentfretin]

Define a window.absoluteURLForAsset function to change the hubs domain by a relative url while loading the referenced url in the hubs extensions. I added

const absoluteURLForAsset =
  window.absoluteURLForAsset ||
  ((asset) => {
    return asset;
  });

in components/environment-settings.js but it probably should be moved in utils.js.

[jywarren]

This would be amazing as well. Would it be spatialized? Ty!

[vincentfretin]

Yes of course it would be spatialized. This is even the default for the aframe sound component https://aframe.io/docs/master/components/sound.html

[uhunkler]

I can work on the audio implementation.

[vincentfretin]

About the absoluteURLForAsset function, the idea for example for outdoor festival scene is to write something like this in a script tag before loading gltf-model-plus.js :

<script>
window.absoluteURLForAsset = (asset) => asset.replace("https://uploads-prod.reticulum.io/files/", "https://cdn.jsdelivr.net/gh/c-frame/outdoor-festival@e9311cf/")
</script>
<script src="../../dist/gltf-model-plus.min.js"></script>

[uhunkler]

I started to work on the audio/sound implementation. There are two central issues.

One is to handle the AudioContext and the necessary user interaction, which I somehow solved. The sound is not auto started but started after a user click. The sound is playing but there is still an issue with the sound component producing the »The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.« notification in the browser. I didn't find out why.

The second is based on combining audio with animation. When Hubs audio is attached to an animated item and the item is in a parented item structure »createEntityAndReparent(node);« on the audio node breaks the parented structure and the item is not animated any more. @vincentfretin, do you have an idea how to solve the parented structure issue. It will be quite common when sound is attached to animated items. I think the FakeEntity for the audio does not work because the sound needs to be started after the user interaction and therefore the sound component needs to be accessible.

Possible solution that I thought about may be

• »createEntityAndReparent(node);« on the parent node and mark the node somehow, where the audio needs to be attached in a second step.
• In the GLTF I tested the parent element comes after the child nodes with the audio component. Therefore parsing as done actually seams not to work.
• The issue with the parent/child structures could also influence other combined Hubs components.

I created a branch for the audio implementation and committed the actual code https://github.com/uhunkler/aframe-gltf-model-plus-uh/tree/audio-inflator The sound source should rotate with the wired »ghost« item. Both items are siblings and are parented to an empty in the center which holds the rotation animation.

[vincentfretin]

For the AudioContext, that's easily solvable with https://github.com/networked-aframe/naf-janus-adapter/blob/master/examples/js/audio-system.js#L126-L162 You can include

  <script src="https://cdn.jsdelivr.net/gh/networked-aframe/naf-janus-adapter@3.2.0/examples/js/audio-system.js"></script>

this should fix the issue completely. This also enable echo cancellation for WebRTC audio.

For hubs component on different parent child. I struggled in a similar way with billboard on parent mesh, and link on child empty, billboard is using fake entity, and link inflator actually get the parent mesh, see https://github.com/c-frame/aframe-gltf-model-plus/blob/be38b88d80362648012808be70ac5b82ca657cfe/src/inflators/link.js#L5-L7 and create an entity, but yeah it just solved this specific case.

I'll look at your example later.

[uhunkler]

I implemented the audio-system.js script and used a FakeEntity for the audio component. The audio component in Blender I have attached to a child Empty of the speaker object. That constellation seams to work. I'll do more testing and commit. I hope you may not need to investigate in the moment.

[uhunkler]

I created a new branch and with this approach the audio/animation combination seams to work. https://github.com/uhunkler/aframe-gltf-model-plus-uh/tree/audio-inflator-alternative-2

[uhunkler]

Today I created a new branch »audio-inflator« based on »main« for the PR.