Rework how audio is managed

[provencher]

Describe the problem

Currently audio clips are added in an ad hoc way that is different for each interactible element in the MRTK. The slider has a script that is programmatically associates the clips to the script, while most other elements use the editor events to trigger audio clips.

With this system a number of problems arise:

• Since the audio is set in the editor event response, it is impossible to programmatically change which clips come out of elements, without painstakingly updating editor references.
• It is impossible to setup an interactible component purely programmatically, without instantiating a prefab, as there is no audio field to address.
• It is not clear what interaction events trigger audio clips, as the events sometimes span across different components.

Describe the solution you'd like

My ideal system would have two sets of scriptables objects

• Audio clips are wrapped in audio config scriptable objects that have all the audio source properties they need to play properly. This would allow each clip to have associated volume, pitch, etc.
• Buttons and other interactions, would have an interaction audio config scriptable object, which would use the audio clip configs in a clear field for each related event. (PointerEnter, Exit, Down, Up, etc).

With a system like this, each object manipulator, button, etc, would have a field for the Interaction Audio Config, that could be set via code, or in the inspector, and it would update in one swoop all the clips used for that given interaction. Then, when iterating on audio, one could update that single config to change the audio for all buttons in a scene, without modifying each game object manually.

Additional context

I'm currently working with an audio designer to expand on the sound library of MRTK, for use in MRTK-Quest.

[jasonhbartlett]

This seems a great idea. I have been struggling with how button that trigger hiding itself on clicking don't have a chance to play the full clip. Looking for solutions to that as well.

[provencher]

Is there any activity on this issue? It remains a pain point for me.

I’ll add that having the ability to just get audio raised events would be great in case I want to have my own audio sources or a different audio system altogether.

[anastasiadevana]

Agreed that this is really painful. In addition, some people might be using a separate audio middleware (Wwise, FMOD, Master Audio, etc) which handles all audio.

I'd love to see some kind of audio handler system, where a developer can choose to completely take over all audio generation, and can just react to events in a uniform way (which action was performed and at which transform / game object).

[Zee2]

Keeping this issue to track our AudioManager story. @davidkline-ms, what is the update on this story?

[IssueSyncBot]

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all-new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visit https://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!