search

monome / norns

norns is many sound instruments.
http://monome.org
GNU General Public License v3.0
621 stars 144 forks source link

softclock #1280

Closed tyleretters closed 3 years ago

tyleretters commented 3 years ago

here's a first draft of softclock, based on @catfact's mockup. usage here.

tyleretters commented 3 years ago
tehn commented 3 years ago

yes, drop in lib and add to startup.lua if it's going to be a global.

i like the usability introduced here.

i'm slightly concerned that we have too many clock approaches:

is there any sense to consider unifying these in any way?

@artfwo any perspectives on this?

artfwo commented 3 years ago

clock provides the coroutine-based interface, that enables multiple entry points in functions (via sleep/sync calls with arbitrary multipliers of the base tempo). clock can also sync to midi, link, and the internal metronome.

Both metro and softclock (i've only briefly looked at the latter) provide callback-based interfaces. i don't know the benefits of callbacks, but the scripts that use either of these two modules will require additional state data and logic for the callbacks.

For instance, the softclock example linked above looks very terse with nothing but print statements in the callbacks. However, using this approach for something less trivial (looping musical phrases) will require additional state variables (such as playhead positions) in the global namespace - a problem that clock tries to solve by encouraging using coroutines to the full extent.

The additional benefit of using coroutines is having multiple sync points inside a single function, which allows to describe musical processes in a really elegant functional way. In some cases it can also make the code more intuitive.

That said, I'd suggest deprecating metro, since clock can do 100% of what metro can do and much more. As for softclock, I don't really understand what problem it solves. It looks like metro using clock for the sync capabilities. @tyleretters what's the use-case for it?

I also see a minor problem with the API design of softclock - string identifiers for subclocks, e.g.

my_clock:add("b", 1/2, function() print("half notes") end)
my_clock:toggle_subclock("b")

instead of:

b = my_clock:add(1/2, function() print("half notes") end)
my_clock:toggle_subclock(b)

In the former case, different string identifiers (in case of typo, etc.) will make the error really hard to debug. Using variables will make it easier.

simonvanderveldt commented 3 years ago

That said, I'd suggest deprecating metro, since clock can do 100% of what metro can do and much more.

clock.sleep is sort of the equivalent of metro, right?

From my testing metro is more stable with regards to the fps/call intervals than clock. I can share some metrics if there's interest.

I don't really understand what problem it solves.

AFAIK clock doesn't work if you want to keep multiple clocks in sync, they all drift. So the approach that most people are taking has been to have one main clock running at the fastest possible rate you need and then do the syncing/divisions of tracks (or whatever you want to sync) within the main clock loop. This also solve the issue of changing the tempo which doesn't work with clock either because the coroutine is suspended/will only be called somewhere in the future it takes too long for it to respond to tempo changes.

tyleretters commented 3 years ago

v1.0.3

https://github.com/tyleretters/softclock/commits/main/lib/Softclock.lua

is there any sense to consider unifying these in any way?

i'm not sure. i saw this as an additive tool like MusicUtil. @dndrks may have had some ideas of making it more core-ish?

is it time to expunge BeatClock, minimally from the docs? dunno how much is built on this...

it is possible these features could be added to clock, instead of a new module.

what's the use-case for it?

primarily, syncing multiple clocks is to the norns internal clock. also drift, as @simonvanderveldt articulated. tertiary, is ease of use and approachability for new users. with Softclock i can now:

function init()
  step = 0
  my_clock = Softclock:new(1)
  my_clock.advance_event = function()
    step = step + 1
    print("already playing quarter notes synced to the norns clock and i'm on step number:", step)
  end
  my_clock_id = my_clock:run()
end
tyleretters commented 3 years ago

i am also curious to have @catfact 's input on this issue

artfwo commented 3 years ago

clock.sleep is sort of the equivalent of metro, right?

To a certain extent yes, it's a functional (pun intended) equivalent.

From my testing metro is more stable with regards to the fps/call intervals than clock. I can share some metrics if there's interest.

Are you comparing metro to coroutines with clock.sleep or with syncing? Actually, even when syncing with the internal clock source, beats are counted with the same underlying delay function as in metro (clock_nanosleep), so I'm curious to see the metrics and the benchmark indeed.

AFAIK clock doesn't work if you want to keep multiple clocks in sync, they all drift.

As this seems to be a recurring misconception, I would like to point out again that there are no multiple clocks when using the clock module. There's a single "reference clock" (a beat counter updated by currently selected source) and a coroutine execution scheduler which can schedule coroutine resumes in beats. So I am assuming you actually meant multiple coroutines here :)

Coroutines aren't deliberately synchronised, as they're supposed to run independently from each other, but they also never drift by design. Roughly speaking, each sync call will schedule resuming of the current coroutine with regards to the aforementioned reference and each subsequent call will "re-adjust" to the new reference position in time, so there will never be any kind of drift accumulation.

With a stable reference (internal or link) this approach works well. With midi, the source is highly unstable and parallel long sync calls may not resume together because of it.

So the approach that most people are taking has been to have one main clock running at the fastest possible rate you need and then do the syncing/divisions of tracks (or whatever you want to sync) within the main clock loop.

The approach is completely fine, as it also facilitates certain script design patterns. But it is not a proper solution to the issue above. It also requires defining a fixed PPQN value which limits the number of available clock divisions, e.g. with a PPQN value of 96 you can divide the beat by 3, 4, or 24, but not by 5 or 7. While this limitation is, again, fine to have on the script level, it is something I would rather avoid in a core library.

The proper solution would be to track elapsed beats in the coroutine execution scheduler and request coroutine resuming only when the expected beat time is reached on the reference "axis". It will be possible with the single threaded clock implementation, I hope to find some time for hacking on it during these holidays, but, anyone, please feel free to contact if you'd like to help :)

With a certain degree of code hygiene (no blocking i/o in coroutines, etc.), clock should be a good API for most cases, as long you don't need sample-accurate scheduling, which is not possible in scripts with the current platform specifics anyway.

This also solve the issue of changing the tempo which doesn't work with clock either because the coroutine is suspended/will only be called somewhere in the future it takes too long for it to respond to tempo changes.

How frequently do you have to change tempo abruptly when performing or recording? :) Anyway, I think this particular problem is also going to disappear with the single-threaded clock implementation.

Both issues will be fixed for good :)

it is possible these features could be added to clock, instead of a new module.

It's possible, but given the completely different design, I would rather keep it in a separate module.

what's the use-case for it?

primarily, syncing multiple clocks is to the norns internal clock. also drift, as @simonvanderveldt articulated. tertiary, is ease of use and approachability for new users. with Softclock i can now:

function init()
  step = 0
  my_clock = Softclock:new(1)
  my_clock.advance_event = function()
    step = step + 1
    print("already playing quarter notes synced to the norns clock and i'm on step number:", step)
  end
  my_clock_id = my_clock:run()
end

I have tried to address the issues of syncing multiple clocks and drift in the answer to Simon above. TL;DR: it's fixable with the current clock design, a single-threaded scheduler will be more stable and react to tempo changes more smoothly.

Regarding usability, I would like to highlight a few advantages that coroutines provide. The code example you shared is a good illustration of state outside of callback function that I mentioned previously. With coroutines, you can implement it in the following way:

function init()
  clock.run(function()
    local step = 0

    while true do
      clock.sync(1)
      print("already playing quarter notes synced to the norns clock and i'm on step number:", step)
      step = step + 1
    end
  end)
end

Note, that in the latter example, step is declared inside the coroutine. Basically you can keep the entire state of this particular sequencer in the scope of the function. If your script is sequencing multiple concurrent processes, each coroutine can be spawned from the same function. Explicit declaration of the while loop also contributes to code readability.

There can be multiple sync points in coroutines too, e.g.

function a()
  clock.sync(1)
  print('after')
end

function b()
  print('before')
  clock.sync(1)
end

function c()
  print('note_on')
  clock.sync(1)
  print('note_off')
  clock.sync(1)
end

function swing()
  clock.sync(1)
  print('kick'); print('hihat')

  clock.sync(2/3)
  print('kick')

  clock.sync(1)
  print('kick'); print('hihat')
end

And sync durations may vary:

dur = 1

function seq()
  while true do
    print('bang')
    clock.sleep(dur)
  end
end

function key(n, s)
  if s == 1 then
    dur = math.random()
  end
end

The above examples can be fully implemented with metro and partially with softclock (i'm not sure how it handles one-shot callbacks), but I think that coroutine syntax results in more coherent and less coupled code, especially as your scripts need more flexibility with time (think rhythm breaks, pattern recording and playback, etc.).

tyleretters commented 3 years ago

zooming way out, the main motivation of this module is ease of use. i want to get to making music on norns. if i were to reverse engineer a user story for it might be:

as a user of norns i want to quickly setup a reliable clock that "just works."

softclock is an attempt to abstract away lots of the stumbling blocks.

How frequently do you have to change tempo abruptly when performing or recording?

not when performing or recording but it happens a lot when writing.

It also requires defining a fixed PPQN value which limits the number of available clock divisions, e.g. with a PPQN value of 96 you can divide the beat by 3, 4, or 24, but not by 5 or 7.

you can set the PPQN to anything and instantiate as many softclocks as you wish. you can also change the PPQN on the fly.

While this limitation is, again, fine to have on the script level, it is something I would rather avoid in a core library.

what is the drawback of doing this? why avoid it in a core library? 96 PPQN seems to be something of a standard. we sorta have the curse of too many choices right now. i see it as a kindness to remove this choice from users, set a sufficiently fine grained PPQN for them, and optionally allow them to increase/decrease.

The code example you shared is a good illustration of state outside of callback function that I mentioned previously. [...] Basically you can keep the entire state of this particular sequencer in the scope of the function. If your script is sequencing multiple concurrent processes, each coroutine can be spawned from the same function.

this is straw man. consider a script that script is a sequencer. for scripts of anything above rudimentary complexity you're going to need to leak/share state from the clocks somewhere.softclock has a both my_clock.transport, and subclock.phase which can be externally called if you want to go the other way.

but i'm not fully understanding you're argument:

function c()
  print('note_on')
  clock.sync(1)
  print('note_off')
  clock.sync(1)
end

what are these doing? i don't understand how this would be used. namely i thought clock.sync()s had to be inside a while loop.

tehn commented 3 years ago

i think what's going on here is a difference in style/preference, and perhaps the fact that coroutines are different than anything else in the norns ecosystem.

once coroutines are understood, they are immensely powerful--- ie, clock.sync() definitely doesn't need to be inside a while loop. @tyleretters if you haven't yet checked out the clocks study, it demonstrates the various rad ways the clock/coroutine system can be used: https://monome.org/docs/norns/clocks/

i see the softclock proposition as sort of helper library, but i fear that it actually obfuscates how much more powerful and interesting the clock coroutine system is... and on the other hand, how easy it is to add on/off toggles and subdivisions to a master clock.

my proposal is to make one additional example for the clocks study which demonstrates how to do exactly this: have a master clock with some sub-clocks and toggles for "tick" functions.

i don't necessarily see a reason not to deprecate the metro library, just to unify the approach. however, it'd mean breaking a lot of existing scripts... though we could make a temporary fake-metro which wraps the clock lib (?) but then the question becomes, when do we pull that out? maybe this is more an issue of revising the documentation and studies so metro is removed. @catfact

catfact commented 3 years ago

ok, question 1: why "softclock." i posted a version of this thing just as a demo / illustration of some statement i made like, "sometimes it is advantageous to drive several software timers with integer divisions of a single master tick; and also, you can do this even with nominal non-integer divisions by accumulating phase remainders." i hope those statements are pretty uncontroversial... this approach is simple and reliable for pattern-based music, its immune to rounding error, and a little helper like this makes it ergonomic.

does it need to be a library module? IDK. i made my initial comment off-handedly thinking both the motivation and method was obvious. but it was non-obvious enough that a code example was illuminating even for a seasoned developer.

but in any case, this helper is definitely not "another clock." it's a soft-timer abstraction that can be driven by any clock.

question 2: deprecate metro? from a pure API design perspective, i agree. but under the hood, metro is substantially lighter. i haven't done timing benchmarks to verify, but @simonvanderveldt's observations aren't surprising.

the heuristics and execution in clock.h/.c aren't totally obvious to me. but it does seem at first glance that there are considerably more opportunities for lock contention in clock. than in metro.c.

in the latter, the only uses of a lock are:

in clock.c on the other hand, there is quite a bit of work done while holding reference.lock. (clock_schedule_resume_sync, clock_get_tempo, etc.) if i understand the design correctly, these functions are called by multiple coroutine-backing threads, to perform synchronization. so there is plenty of opportunity for contention.

in any case, having routines to compensate for drift between coroutines (by effectively adding jitter), while definitely useful, is not really the same as simply eliminating drift in the first place by using a single thread.

obviously some future single-threaded implementation of clock would change the equation. but that's not what we have today.

in a nutshell, the requirements for clock are much more complex than for metro, its a much heavier system, and the synchronization routines will introduce more jitter and drift via lock contention, the more clocks are sharing a reference. so although it is certainly possible to replace any use of metro with clock, i don't think its always a good idea when there is no desire for different timer routines to be synchronized.

oh, just real quickly:

How frequently do you have to change tempo abruptly when performing or recording?

i don't think there is anything remotely unreasonable about wanting to perform extreme tempo modulation to get a musical effect.

artfwo commented 3 years ago

yes, by deprecating metro i don't mean removing it of course.

and let me once again emphasise that there is no drift or any kind of drift compensation in clock by design. basically, each sync call will schedule next coroutine wakeup in n seconds depending on what is the current beat count at the reference (with some lookahead logic).

this design basically removes the necessity to compensate for drift, since any scheduling mistakes won't accumulate, because the next sync call will use an up-to-date reference data for subsequent resumes.

oh, just real quickly:

How frequently do you have to change tempo abruptly when performing or recording?

i don't think there is anything remotely unreasonable about wanting to perform extreme tempo modulation to get a musical effect.

i agree 100% and current clock implementation still allows for this, as you can always explicitly restart or re-enter the script coroutines whenever you're modulating the tempo intentionally.

catfact commented 3 years ago

thanks for clarifying.

the next sync call will use an up-to-date reference data for subsequent resumes.

.. but this is precisely what i would call a "drift compensation" mechanism. the sleep period is recalculated on every tick based on the actual running time of the reference CLOCK_MONOTONIC. this is adding jitter to eliminate drift.

i have no problems whatsoever with this scheme, but it's not perfect; especially so when the requested sleep periods are floating point and don't share a common factor. it also inevitably requires some extra work between the thread being woken, an event being posted, and the thread sleeping again. this work produces some jitter and/or drift. when many coro threads are running, the work is gonna take longer...

so it's a tradeoff: you get the coroutine interface, but per-beat timing is sometimes wonky.

i made a small test script, imagining ways of making a drum-machine-style pattern player.

here's a gist:

https://gist.github.com/catfact/b291e43cfbd9ad7f2ce23334c52cddbb

there are three different scripts: one using coroutines on the internal clock, one using a soft-timer on the internal clock, and one using a soft-timer on a metro.

i can't for sure say there is an audible difference between the two timer sources; suspect the metro is a little tighter but need to take better measurements.

however, the coroutine version has issues:

artfwo commented 3 years ago

@catfact i don't have access to the hardware at the moment, so can't reproduce the issues you have, but does the skipping problem occur if you change the threshold from 2000 to 200 or 100 at line link?

catfact commented 3 years ago

at first blush, changing the divisor to 100 does seem to fix the sporadic skipping.

i'm still confused as to why the kickdrum pattern is yielding x...x...xxx. instead of x...x...x...x..x as expected, and the "hihat" is yielding x.xx. instead of x..x..x.

(again, must be user error, but it is still surprising to me...)

catfact commented 3 years ago

uh real quick

@tehn

clock.sync() definitely doesn't need to be inside a while loop.

in the context above, it does. we're talking about infinite sequences...

tehn commented 3 years ago

in the context above, it does. we're talking about infinite sequences...

right, i was responding to what i likely mis-perceived as general confusion re: coroutines

catfact commented 3 years ago

ok, let's take a HUGE step back for a second and look at the big picture:

i think this module is useful.

the basic structure is not a clock source; it's a nice way of patterning multitrack sequences where (1) durations are specified as arbitrary ratios, (2) the master tick rate is also arbitrary (but is presumed to be at steady rate for this implementation.)

the main thing that the structure manages is the fractional phase of the sequences. not very complicated but it is useful boilerplate.

the structure can be used in many ways with clock or without:

i think maybe the objection/confusion lies in naming it a "clock" and structuring the API as if it were a clock source. it could be called something with "timer" or "sequencer" instead?

also of course open to suggestions for restructuring the callbacks however.

tehn commented 3 years ago

i think maybe the objection/confusion lies in naming it a "clock" and structuring the API as if it were a clock source. it could be called something with "timer" or "sequencer" instead?

this, hugely.

catfact commented 3 years ago

my other thought is: although i think this is a useful technique, i'm not sure this form of it hits the right level of utility to justify a library module.

e.g., @tyleretters i'm looking at the song repo, and it doesn't seem to really require the fractional phase trick. it just operates on 16th-notes and could have been done with a simple integer-counting structure like my patterns example. so for that use-case i think this is an unnecessary abstraction. (OTOH - hey, maybe you want to modulate each track's "virtual tempo" independently.)

and on the other hand: if i were to use something like the original mockup i posted (containing 17ths and other weird divisions,) it would probably be in a musical context where the next thing i would want is to specify per-stage durations (like in my pattern script above.)

this would be to implement something analogous to 251e sequencer logic, or to play arbitrary timed patterns with a MIDI clock. so for that set of use cases, the abstraction is not general enough - but the use-case is specific enough that i start to question the need for the abstraction at all.

tyleretters commented 3 years ago

totally onboard with not calling it a clock. that solves so many problems! the hardest things in computer science...

@catfact softclock was a bazooka to the fly that is song. it started off with wonky stuff but those were quickly cut. the original thrust of this whole "sequencer-softclock" endeavor was getting something going for yggdrasil, so each track could have whatever insane timing needed. this lead to my ppqn explorations which you then quickly improved with your original softclock gist.

i'll take a todo to rebrand this as something sequencer-y, rework the api to match, and open a proper pr. thanks for all the feedback and energy on this :)

i think we're good to close this issue unless anyone wants to discuss anything else about it.

artfwo commented 3 years ago

i'm still confused as to why the kickdrum pattern is yielding x...x...xxx. instead of x...x...x...x..x as expected, and the "hihat" is yielding x.xx. instead of x..x..x.

@catfact, so this seems to happen because the pattern {4, 4, 4, 3, 1} is defined in intervals, but as the values are used for sync, they're interpreted as "next multiple of", so you'll get kick drum at the following beats at some point:

{ 20, 24, 28, 30, 31 }, { 32, 36, 40, 42, 43 }, { 44, ... }

and the part with kick at 30, 31, 32 is heard as xxx.

i've raised a PR with sync threshold multiplier tuning here: https://github.com/monome/norns/pull/1287

catfact commented 3 years ago

ok, i see... so this is a pretty important thing to understand about clock.sync(), and i'm not sure the documentation conveys it right now. (says: "clock.sync sleeps until the next subdivision specified arrives," which i find a little hard to interpret.)

when i say clock.sync(x), it doesn't mean "sleep for duration of x times the tempo quantum" (as it would with TempoClock in supecollider, for example) - instead it means, "sleep until the clock's beat counter reaches x * N for some integer N"

is that right? it strikes me as a little bit challenging to reason about for units of >1 beat. (given that it is potentially dependent on when you started the coroutine, right?)

artfwo commented 3 years ago

yeah, calling clock.sync(1.5) when you're starting the coroutine sometime around beat #2 will resume the coroutine at beat 3. calling it again around beat 3 will resume at 4.5.

when i say clock.sync(x), it doesn't mean "sleep for duration of x times the tempo quantum" (as it would with TempoClock in supecollider, for example) - instead it means, "sleep until the clock's beat counter reaches x * N for some integer N"

N will always take the smallest possible value, so that x * N > current_beat() is true.

upd: the docs could be improved for sure, do you have a better idea for describing sync behaviour?