Stream params automation

[dy]

Induced by https://github.com/audiojs/audio-gain/pull/9.

Many streams take in params for processing:

• gain takes volume
• biquad takes frequency, Q, type
• generator takes function
• oscillator takes frequency
• etc.

How do we change these params during processing?

There are a couple of points to consider. First off, param value can be fixed for a frame (a-rate) or per sample (k-rate). In case of gain, fixing volume for a frame may cause noticeable steps if volume changes significantly next frame, because unlike biquad filter, it has no "elasticity". At the same time making it k-rate would cause overhead if user liked to smoothly fade-in or fade-out, for example. It would force him to automate each value of volume manually.

Web-Audio-API guys use AudioParam for that, and there is nice pseudo-audio-param for that purpose (it is tiny!). It may solve the problem of setting/automating params, as well as add smoothing to their change, which would make audio behave more naturally.

How should we expose params?

We could set them as a returned function’s property, like now it is done with end method:

const Osc= require('audio-oscillator');
const Gain = require('audio-gain');
const Filter = require('audio-filter');
const Speaker = require('audio-speaker');

let saw = Osc({ frequency: 440, type: 'sawtooth'});
let gain = Gain({ volume: .8 });
let filter = Filter({ frequency: 520, type: 'lowpass', Q: 1});
let write = Speaker();

//start streaming
(function nextFrame () {
    let buffer = gain(filter(saw()));
    write(buffer, nextFrame);
})();

//API proposal
gain.volume; //returns AudioParam
gain.volume.value = .9; //sets AudioParam value
gain.volume = .9; //we can make it through setter
gain.volume.setValueAtTime(.9, 1.25);
//...other pseudo-audio-param methods

The problem with this approach is that it forces us tracking time within streams. It would be nice to avoid some audio-context mediator, following "independent" packages approach. Audio-generator, for example, tracks time from the moment of the first frame sent. We can set that time property straight on AudioBuffer.

Another possible option - passing AudioParam instance to options of a stream. In that case components do not depend on pseudo-audio-param and can operate in "static" mode, like they do now. If user wants to change some param, he should pass AudioParam instance.

Alternative way

Third, and really easier way - just passing a function as a param value to options, like

Gain({ volume: () => volume });

That leaves time tracking, automation etc up to user.

Really appreciate your thoughts guys @audiojs/devs

[ahdinosaur]

hmm...

i like the idea of being able to pass in either a parameter value or a "stream of parameter values". so in a pure functional approach the "Alternative way" makes the most sense, it's basically a source stream. but if we think AudioParam is a good way to be a source stream i can imagine we accept that as a parameter too.

@mmckegg do you have any thoughts here?

[mmckegg]

Have had nothing but problems with AudioParam in Loop Drop. I don't think it solves the problem very well.

I would suggest a setter that accepts Number values or Stream, and adapts accordingly. If you passed it a stream, it would buffer up frame lengths and pass this to the audio function (then the function can choose k-rate or a-rate). You'll be able to use existing audio generator/transforms and plumbing to create the automations. The only shared state (or implicit knowledge) needed would be the sample rate.

I am pretty sure using the sample rate for synchronisation would work, but yeah, without a central clock, things could be a little floaty and non-deterministic... needs to be experimented with.

[dy]

That’s interesting - to use other audio sources for creating params automation, that allows for easy modulations. In that, params may take sync source form, like audio-oscillator, -generator etc. I guess if we add audio-adsr and audio-sequence/MIDI stream, we cover most of the modular synthesis cases. Like

let lfo = Sine({
  frequency: 1.08
});
let osc = Sine({
   frequency: 440,
   detune: lfo
});

//...pipe oscillator to output etc