WebAudio API concerns from HbbTV liaison letter

[jpiesing]

CTA has received a liaison letter from HbbTV noting the following concerns about the way in which the WebAudio API is included in WMAS 2018.

The Web Audio specification is a requirement from WMAS. We notice that you already have two exceptions due to dependencies on Media Streams. We have a significantly smaller subset of the Web Audio API in our specification already. Some concerns have been raised that your subset may have increased hardware requirements compared to our subset when run on a media device, particularly for multi-channel audio. Some examples include the AudioNode processors like DynamicsCompressorNode, BiquadFilterNode, MediaElementAudioSourceNode and more. Even if these don’t need specific hardware acceleration, concerns have been raised that reliable realtime performance may impose processor requirements that may not be continuously achievable. Do you have an opinion?

[JohnRiv]

We should look into additional exceptions we can include related to Web Audio.

[jpiesing]

I see two dimensions to a discussion about the Web Audio API.

• One is which interfaces are supported (i.e. work - not just are present).
• The second one is which audio codecs are supported

For the interfaces, here is a list with the ones included in the HbbTV subset of Web Audio called out. As you can see, there's a huge difference. Please note that the HbbTV subset was inspired by subsets used by others.

• AnalyserNode
• AudioBuffer - length, sampleRate, duration, numberOfChannels
• AudioBufferSourceNode - buffer, start, stop, onended
• AudioContext - createBuffer, createBufferSource, destination, sampleRate, currentTime, decodeAudioData
• AudioContextOptions
• AudioDestinationNode - maxChannelCount
• AudioListener
• AudioNode - connect,disconnect, context, numberOfInputs, numberOfOutputs, channelCount, channelCountMode, channelInterpretation
• AudioNodeOptions
• AudioParam
• AudioProcessingEvent
• AudioScheduledSourceNode
• AudioWorklet
• AudioWorkletGlobalScope
• AudioWorkletNode
• AudioWorkletProcessor
• BaseAudioContext
• BiquadFilterNode
• ChannelMergerNode
• ChannelSplitterNode
• ConstantSourceNode
• ConvolverNode
• DelayNode
• DynamicsCompressorNode
• GainNode
• IIRFilterNode
• MediaElementAudioSourceNode
• MediaStreamAudioDestinationNode
• MediaStreamAudioSourceNode
• OfflineAudioCompletionEvent
• OfflineAudioContext
• OscillatorNode
• PannerNode
• PeriodicWave
• WaveShaperNode
• StereoPannerNode

For codecs, there may be a big difference between supporting some simple stereo formats and supporting multi-channel formats as found in the WAVE content spec.

[JohnRiv]

Further research to do:

• Any other published subsets to pull from?
• Review Web Audio API spec and compare to above list
• For codecs, can we come up with a generic statement so we don't have to duplicate any list of codes that the Content Spec publishes?

[JohnRiv]

Review Web Audio API spec and compare to above list

The Web Audio API spec defines 77 interfaces (or at least interface-like things, I didn't check to verify every one is an interface), compared to the list of 36 above

[jpiesing]

This is no longer a blocking factor for HbbTV. For HbbTV 2.0.3, we will stick with our existing subset of Web Audio and update the reference to the same as WMAS 2018. We lack the expertise and time to properly address the issues with porting / integrating the Web Audio API into a TV / STB / media device platform. That doesn't mean the work doesn't need doing and we may return to it in the next iteration of our spec.

[JohnRiv]

On the 2/19/20 HATF call, we decided to close this for now for the reasons @jpiesing mentioned above.