Closed catfact closed 4 years ago
I took a stab at a lua-side clock here (in dust, maybe it should be here instead.) https://github.com/monome/dust/pull/135
Lua seems to work pretty well for 24ppq, but rebroadcasting external clock to other connected devices might be better done on the native side.
Here's a draft straw-man proposal for a native clock system. This is going to be simultaneously inappropriately maximalist, poorly informed, and a little obvious, but just to push the conversation:
UI:
Lua/script layer:
clock.setBPM
, clock.start/stop
, clock.reset
(or similar) that scripts can use to control master bpm and reset beat/phase with knobs/buttons/params. (setBPM
, start
, and stop
would be ignored if the master clock source is midi.)matron/C layer:
SuperCollider layer:
Re: Ableton Link, after thinking about this some more, and catching up on the discussion on the SC Link pull request, I lean towards relying on the SC implementation of Link once it lands (hopefully in 3.11?) https://github.com/supercollider/supercollider/pull/3351
As an experiment, though, I'm throwing the Link library into the C layer to see if I can work Link into the existing BeatClock system.
pinging this thread regarding link support.
A report from a colleague:
SuperCollider 3.10.1 successfully built for RaspberryPi… …and the still-in-development Ableton Link code, also working on the Pi!
I've started some of the prep work for Link support, but it touches a lot of things that are changing in 2.0, so I've put that work on hold until after 2.0 is finished.
Still on the fence about whether it makes more long-term sense to use the SC Link implementation or do a direct integration of the Link C library into matron. In the current state of the system, timing is a responsibility of the scripting world.
fwiw, i think it would be better to put Link support in the C components if possible. we're moving further in the direction of making norns ecosystem independent of supercollider; SC becoming one of many options for DSP engine interfacing with the scripting / timing layer.
see also PR 712 https://github.com/monome/norns/pull/712
which makes steps to support lua code execution as coroutines, with worker threads under the hood.
aye, the clock module support for syncing to external events is in progress. link should fit perfectly as long as it's implemented in matron.
I saw this pull request and from some recent experiments with particle synthesis I can see big benefits to an approach that gets clocks off the main lua thread.
I’m trying to run some metros in the 100+ calls per second range and it’s interfering with the menu system running smoothly and my ability to update the script via Maiden. (And I don’t think it’s what’s executing in that callback, because CPU is reporting ~17%.)
@nathankoch while i agree that this is a good idea (servicing clock callbacks from worker thread), i also wonder if this isn't the event queue itself getting bogged down.
the CPU reporting on norns is average of all 4 cores, so isn't really granular enough to see if things are stalling in matron
's event queue servicing. this is a general area that will get a lot more close attention for 2.1.
ii am surprised that maiden
is affected though... what is in the callbacks? if you are printing lots and lots of stuff, then maiden
will be having to deal with, i dunno, constantly updating the layout of the REPL or something... getting a bit OT there.)
I’ll leave my (terribly un-optimized, with a ton of garbage collection from these particles being spawned and killed) code here: https://github.com/nathankoch/NornsMachines/blob/master/grains/simplegrains.lua
(It’s basically a lua port of standard Processing/oF style particle systems as found in Daniel Shiffman’s Nature of Code.) What I’ll probably do is dig into this more, do some optimizations and isolate the problem better in fresh code - then create a separate issue.
(I don’t see any rogue print statements but I can’t rule out one slipping in. I know for a fact that tons of printing stalls out/overwhelms maiden
‘s REPL but I’d call that expected behavior.)
Good to know also that the CPU is reporting across all cores.
I know for a fact that tons of printing stalls out/overwhelms maiden‘s REPL but I’d call that expected behavior.
known (and unfortunate behavior). i could have sworn we had a bug tracking this but can't find it... @ngwese?
I wonder if it would be worth investigating a solution like what Unity 3D’s console does, where multiple identical prints to the console (or multiple identical errors getting thrown) just get an incrementing number tacked next to one line.
(But I’m getting way OT from this original issue here)
@artfwo finally took a look at your coroutine clock PR #712. I'm about ready to start working on Ableton Link again. If I keep working on the plan described in https://github.com/monome/norns/issues/129#issuecomment-405038164, will that be compatible with what you're doing, or does the plan need to change? (Along with the obvious changes for post-2.0 UI.)
@Dewb not exactly, the clock module already provides beat syncing coroutine execution scheduler with arbitrary multiplication and division. Currently it only has one clock source - tempo, MIDI and link are in progress.
To push the clock forward you'll need to update the master clock rather frequently using the "app" (thread-safe) link methods. And then call clock_update_reference_from(<beat>, <tempo>, CLOCK_SOURCE_LINK)
. The API will provide a method to select source eventually. I have also started some groundwork on the link bindings for C here (please feel free to send code): https://github.com/artfwo/link-c
And here's a crude matron clock source for link (not currently pushed because it's a horrible hack, please also feel free to contribute): https://gist.github.com/artfwo/2de822c5b6389d310299d6f29440a458
And here's a test script for the clock API: https://gist.github.com/artfwo/115a7129b461f3f07fae5016ae15d048
Awesome! Looks like you're well on the way, then, I'll try to build your WIP branch + gists.
For building gists, add link-c
and link
as submodules to a subdirectory in norns and compile ableton_link.cpp
as a static lib or just add it to the list of matron sources - waf will automatically build the object using c++ compiler and link it with the matron binary. Link.hpp
should also be in the include path.
Updating last year's straw man proposal to adapt to 2.0 changes and @artfwo's coroutine clock system.
UI:
(more to come)
I would be excited about all of that.
What can I do to help? (probably in the testing and breaking things dept)
stopping and resetting is currently not possible with the clock module, but is possible to implement in theory - it will require rescheduling sleeper threads when start/stop/reset command is received or there is a significant tempo change, for example.
@dewb I agree, system wide clock is the future.
i am hoping to work on the param menu expansion soon but we can hack in the clock system ahead of that is needed.
(scope creep) perhaps a "transport control" UI widget for start/stop/reset? I could see this being super useful with MIDI and Link when norns is the master clock.
looking at this again as i'm planning to integrate it into my in-process param menu overhaul.
i'll start work on a system-wide clock that (yikes) basically is feature-overblown (due to all of the sync possibilities i guess)
the param submenu system should make this clean and navigable though.
Clock section in params sounds great.
I got a little ways towards prototyping a global clock before putting it on hold when the menu revamp started. Let me know if there’s a section of work that could be broken out that I could help with.
I think the biggest remaining issue in adopting the coroutine system for a global clock is having some sort of cancellation system to retrigger waiting coroutines when the master tempo changes. Right now going from a slow tempo to a fast tempo can give you events much later than expected.
I think this is not a really big issue - abrupt tempo changes from 20 to 200 are rare and you still have enough flexibility from the script to keep the situation more or less under control.
For example, instead of calling clock.sync(100)
it's possible to use a loop: for i=1,100 do clock.sync(1) end
in scripts that expect the tempo to change quite often. Retriggering waiting coroutines is also possible from the script of course.
menu revamp shouldn't block progress on a clock class--- i'm just hoping to get config options for the clock in the menu. all of these should be configurable with class functions as well.
@dewb would be happy to build upon what you've started
(and also, i'm looking to finish this menu revamp quite soon)
if anyone has begun work on this, could you make your branch visible?
there's interest from several people to work on this, but (my lack of) coordinating is probably confusing everyone. i am more than happy to have someone take the lead on this.
this is effectively in place. minor polish and fixes remain.
there are a number of musically useful ways that lua callbacks could be synchronized to various clock sources:
internal, using the
metro
system; these are high-priority threads using nanosleep, should be maximally accurateMIDI clock (24 ppqn)
sample clock; we could get a timestamped bundle from supercollider every N samples; this will have some jitter in the 100s of usecs range, but no drift
ableton link (???)
create a Clock module or something, allowing scripts to conveniently select from these