Are Float32Arrays in inputs, outputs, and parameters arguments to process() reused between calls or created each time?

[karlt]

This was originally filed as https://github.com/WebAudio/web-audio-api/issues/1127 but the resolution is not clear to me.

https://webaudio.github.io/web-audio-api/#rendering-loop has

If this AudioNode has any AudioNodes connected to its input, sum the buffers made available for reading by all AudioNodes connected to this AudioNode. The resulting buffer is called the input buffer.

which implies that the "input buffer" used to "invoke processFunction" is a new object for each invocation. Is that the intention?

https://heycam.github.io/webidl/#es-sequence indicates that IDL sequence<T> types correspond to ECMAScript Array values. Conversion to ECMAScript values produces a new Array object, so the Arrays are new objects each time, if process() is invoked as a WebIDL callback. However, the Float32Array elements of these arrays could still either be reused or created each time, because they are not copied on conversion.

[hoch]

Is this question about AudioWorkletNode or a generic AudioNode case?

[karlt]

This issue was intended specifically for the Float32Arrays provided to AudioWorkletProcessor.process(). This case is special because this runs on the rendering thread, where I expect that minimizing garbage is preferable.

For completeness though, there are two other APIs in the spec that pass Float32Arrays to JS:

let buffer = new AudioBuffer({length: 1, sampleRate: 48000});
buffer.getChannelData(0)["expando"] = 1;
console.log("AudioBuffer.getChannelData() returns " +
            (buffer.getChannelData(0).hasOwnProperty("expando") ?
             "same object" : "new object"))
let context = new OfflineAudioContext({length: 1, sampleRate: 48000});
let a = new Float32Array(2);
a["expando"] = 1;
let shaper = new WaveShaperNode(context);
shaper.curve = a;
console.log("WaveShaperNode.curve returns " +
            (shaper.curve.hasOwnProperty("expando") ?
             "same object as setter" : "own object"))
shaper.curve["expando2"] = 1;
console.log("WaveShaperNode.curve returns " +
            (shaper.curve.hasOwnProperty("expando2") ?
             "same object" : "new object"))

AudioBuffer.getChannelData() provides the same object each time in both Chome and Firefox, though that is not clearly specified AFAIK. The behavior of WaveShaperNode.curve differs between implementations. Neither return the object passed to the setter, but the behavior differs for repeated getter calls. The Firefox behavior for WaveShaperNode.curve was changed at https://hg.mozilla.org/mozilla-central/rev/8895dafe20fea0a2b339a8eeb556bcf4e3ac08ef#l5.23 I don't know the reasoning for the change because it appears to have snuck in with an unrelated change.

[karlt]

https://github.com/WebAudio/web-audio-api/issues/1933#issuecomment-498412673 suspects a previous resolution based on the reasoning that objects being transferred to another thread would be too much of a problem.

I searched old issues and couldn't find such a resolution or discussion. New objects will need to be created if and when buffers are neutered or can't be reused for other reasons, but this doesn't mean that new objects must be created on every call.

The most recent comment I found was https://github.com/WebAudio/web-audio-api/issues/1127#issuecomment-273841804

As a preview of the answer, buffer reuse across calls is likely to be permitted since otherwise there will be performance issues.

[hoch]

This issue was intended specifically for the Float32Arrays provided to AudioWorkletProcessor.process(). This case is special because this runs on the rendering thread, where I expect that minimizing garbage is preferable.

This issue has some discussion, so we certainly discussed it in the design process. (@padenot knows)

We all are aware that this will affect the GC, but it requires a new mechanism that prevents AWGS-generated (provided by audio subsystem) buffers from being transferred to the other realms. That seems like a non-trivial change, perhaps out of Web Audio API's boundary.

Another thought came up was to designate the WASM heap for this purpose, but also not feasible for V1.

[karlt]

Given there are other APIs that don't specify when a new object is created or reused, it seems acceptable to leave this unspecified for processFunction.

I don't think this needs to be resolved for v1. More specifics could perhaps be added when implementations have had more time to optimize.

[padenot]

See https://github.com/WebAudio/web-audio-api/issues/1933#issuecomment-505551737 that is relevant to this discussion.

[padenot]

This is being handled in https://github.com/WebAudio/web-audio-api/issues/1933. The answer to the question asked in this bug is "inputs, outputs and parameters are reused between calls".

1933 doesn't have prose yet, but there is agreement between implementers, and we to test implementation before writing the text. The outcome of this issue will not change however, objects are reused.