Tonejs / Tone.js

A Web Audio framework for making interactive music in the browser.
https://tonejs.github.io
MIT License
13.37k stars 976 forks source link

Automation lanes/events (like in Ableton) #1146

Closed braebo closed 1 year ago

braebo commented 1 year ago

I'm currently stumped trying to create Automation events in a way that is atomic (can be scheduled / rendered offline). The simplest example to recreate would be similar to an automation / modulation lane like in Ableton Live:

Screenshot 2022-11-18 at 2 29 26 PM

Experimenting with different ways to compose these events, we've started with a simple data model:

const AutomationLane = {
  startValue: 0,
  paths: [
    { time: 0.5, value: 1, exponent: 0.3 },
    { time: 1, value: 0, exponent: 3 }
  ],
  endValue: 0
};

The key here is that the lines connecting the startValue, Path values, and endValue both have unique, specific exponents for their curves. We have it working with realtime Midi CC and with raw Midi files via setValueCurveAtTime, but a proper offline/transport-synced solution is alluding me.

A few things we've tried:

Using multiple Signals and Pows gets things going:

const pow1 = new Tone.Pow(3).connect(target);
const pow2 = new Tone.Pow(0.3).connect(target);
const signal1 = new Tone.Signal(0).connect(pow1);
const signal2 = new Tone.Signal(1).connect(pow2);

signal1.rampTo(1, 0.5, 0);
signal2.rampTo(0, 0.5, 0.5);
Screenshot 2022-11-18 at 10 24 40 PM

But their values compound (they go from 1-2 instead of 0-1). To compensate for the compounding, we tried setting each one to 0 when it's done, and setting the next one to the previous one's value like so:

const pow1 = new Pow(3).connect(target)
const pow2 = new Pow(0.3).connect(target)
const signal1 = new Signal().connect(pow1)
const signal2 = new Signal().connect(pow2)

signal1.linearRampToValueAtTime(1, 0.5)
signal1.setValueAtTime(0, 0.5)

signal2.setValueAtTime(1, 0.5)
signal2.linearRampToValueAtTime(0, 1)

But it gives confusing results, including an offset that causes it to over or undershoot 0 and 1:

Screenshot 2022-11-18 at 8 47 31 PM

We tried sticking to 1 Signal and 1 Pow, but scheduling a callback that updates the exponent via transport.scheduleOnce:

let pow = new Tone.Pow(3).connect(target)
const signal = new Tone.Signal().connect(pow)

signal.linearRampToValueAtTime(1, 0.5)
signal.setRampPoint(0.5)
signal.linearRampToValueAtTime(0, 1)

context.transport.scheduleOnce(() => {
    pow.value = 0.1
}, 0.5)

context.transport.start()
Screenshot 2022-11-18 at 3 17 03 PM

It seems to ignore the initial exponent value. If we remove the part where the exponent changes at 0.5:

let pow = new Tone.Pow(3).connect(target)
const signal = new Tone.Signal().connect(pow)

signal.linearRampToValueAtTime(1, 0.5)
signal.setRampPoint(0.5)
signal.linearRampToValueAtTime(0, 1)

// context.transport.scheduleOnce(() => {
//  pow.value = 0.1
// }, 0.5)

context.transport.start()

It looks like this:

Screenshot 2022-11-18 at 8 49 34 PM

I'm fumbling around in the dark here it seems 😅

So far, the system I've come up with for modularizing / generating these works pretty nicely -- I just can't seem to get the actual signal automation right!

For the curious, this is what it looks like as of now: ```ts fn: (target, context) => { const lane: AutomationLane = { target: { pointer: () => {}, min: 0, max: 1, }, startValue: 0, paths: [ { time: 0.5, value: 1, exponent: 3 }, { time: 1, value: 0, exponent: 0.3 }, ], endValue: 0, } const AutomationEvent = (lane: AutomationLane, automationTarget: Signal | number) => { const signals = lane.paths.map((path, i) => { const pow = new ScaleExp({ context, exponent: path.exponent, min: lane.target.min, max: lane.target.max, }) const signal = new Tone.Signal<'normalRange'>({ context, value: 0, units: 'normalRange', }) signal.debug = true // Make sure there is a setValue event at the start of the automation. signal.setValueAtTime(0, 0) // Hook them up. signal.chain(pow, target) return signal }) const event = new Tone.ToneEvent( (now) => { for (let i = 0; i < lane.paths.length; i++) { if (i === 1) { signals[i].setValueAtTime(lane.startValue, now) } const startTime = i === 0 ? now : lane.paths[i - 1].time const startValue = i === 0 ? lane.startValue : lane.paths[i - 1].value console.log({ startValue, startTime }) const path = lane.paths[i] // Do the automation. signals[i].linearRampToValueAtTime(path.value, path.time) // Set the current paths value to 0 when it's complete. signals[i].setValueAtTime(0, path.time) // And set the next paths value to the current path's value. if (signals[i + 1]) { signals[i + 1].setValueAtTime(path.value, path.time) } } }, { context }, ) return { event } } context.transport.start() const automation = AutomationEvent(lane, target) automation.event.start() } ```

Anyways, I suppose my question is:

Is there a recommended approach to solving this problem? Perhaps someone could offer any insight or advice to point me in the right direction? Any feedback would be greatly appreciated!!

Thanks! And thank you to all of the brilliant minds behind this amazing project 🙏

dirkk0 commented 1 year ago

My advice: don't do it like that.

Create an algorythmic but discrete array of values, and ride the sliders with that.

braebo commented 1 year ago

@dirkk0 hmm ok that makes sense, thanks! When you say "ride the sliders", what do you mean? I was imagining that setValueCurveAtTime would be ideal if I understand you correctly.

dirkk0 commented 1 year ago

I didn't try setValueCurveAtTime out, but yes this seems an option.

My use case was 'writing automation lanes in real time' while hearing the stems and then 're-playing them while doing the offline rendering' as fast as possible. What worked for me was to write the values into a vanilla JS array from sliders like a mixing desk (hence 'ride the sliders') with appropriate resolution and writing/reading these values respectively. Simple approach that worked very well for my clients use case.

braebo commented 1 year ago

I'll give that a shot - thanks for the advice!