Standardize SC synth engine commands

markwheeler commented 6 years ago

I think it'd be great to standardize the SC engine interfaces as much as possible for engines which are synths/voices. This would be a simple first step towards making engines interchangeable between scripts.

Here's a proposal to get the conversation started, it's based on some comments @catfact posted on lines a while back and my own testing.

engine.noteOn(id, freq, vel) Start a note, the engine starts a synth as it sees fit. id would typically be a MIDI note number but could be any unique integer to refer to this note in future.

engine.noteOn(id, freq, vel, sampleId) I've also been using this extended variation in Timber for sample playback that lets you specify a sample.

engine.noteOff(id) engine.noteOffAll() Stop a note by id or stop all notes.

engine.noteKill(id) engine.noteKillAll() Stop notes immediately, ignoring their usual envelopes, typically freeing the synth. Useful in a sequencer script when notes have long release times for example.

engine.pitchBend(id, ratio) engine.pitchBendAll(ratio) Bend a note by ratio or bend them all.

engine.pressure(id, pressure) engine.pressureAll(pressure) These correlate to key pressure and channel pressure (after touch) in the MIDI spec.

engine.timbre(id, timbre) engine.timbreAll(timbre) Correlate to MPE spec.

I have an example implementation of this on a branch, SC and lua script here: https://gist.github.com/markwheeler/b88b4f7b0f2870567b55cbc36abbd5ea https://gist.github.com/markwheeler/fbf0d7e62ce15e1d7110bc7e58f4022f

Thoughts:

Obviously not all engines will implement all of these but at the moment they would need the empty functions in SC as I don't think the lua script can check if a function exists before calling it. Is that OK or should we put them somewhere else and have them be overridden?
Would it be good to have a standard modulation input (or several) that engines can implement however they like?

antonhornquist commented 6 years ago

Agreed - standardizing synth engine commands would be good. Not only for voice / polyphonic engines but other use cases, ie. "triggerable engines" like Ack. A set of engine command interfaces that scripts may utilize (require) as an alternative to require a specific engine may be something to consider(?)

engine.noteOn(id, freq, vel) Start a note, the engine starts a synth as it sees fit. id would typically be a MIDI note number but could be any unique integer to refer to this note in future.

I guess id being arbitrary reference implies voice allocation handled SC-side (like PolyTemplate)? The alternative is lua-side voice allocation (like gong which uses exp/voice). The gong approach is id == voicenumber with polyphony predefined in a polyphony command. (gong engine code might be hard to decipher due to its use of CroneGenEngine. The gist is that CroneGenEngine subclasses defining SynthDefs with a gate and freq argument statically allocates polyphony number of voices and exposes commands akin to the ones you've listed)

they would need the empty functions in SC as I don't think the lua script can check if a function exists before calling it

IMO lua-side engine metadata (supplying engine commands, polls, etc without having to load an engine) would help here.

also for onFree to work here I believe you may need to assign a NodeWatcher to the node: https://github.com/markwheeler/dust/blob/poly-template/lib/sc/Engine_PolyTemplate.sc#L57

markwheeler commented 6 years ago

Definitely like the idea of scripts requiring an engine type rather than specific engines. Some thought needs to go into what would be the right interface for an engine that operates as a sampler. I guess for a noteOn equivalent you'd want something like id, sound-id (the sample), vel and optionally a way of either specifying a target frequency or a pitch shift? Gets a little tricky there.

Briefly looking at CroneGenEngine it looks like you're trying to tackle many of the same things here already (I wasn't aware of this class before).

I'm not immediately seeing the advantage in handling voice allocation on the script side - with the aim being to decouple as much as possible doesn't it make sense for things like polyphony and voice stealing logic to stay in the engine where the author knows the performance limitations and character they're after?

onFree seems to be reliably getting called for me, did you find some cases it isn't?

antonhornquist commented 6 years ago

Some thought needs to go into what would be the right interface for an engine that operates as a sampler. I guess for a noteOn equivalent you'd want something like id, sound-id (the sample), vel and optionally a way of either specifying a target frequency or a pitch shift? Gets a little tricky there.

The Ack sample player engine is really simplistic since just has a trig command it and an AR volume envelopes - no noteOn and noteOff commands.

A more traditional sampler engine may need something else, agreed, unless it is based on mapping samples to midi ranges and the matron (lua) -> crone (sc) is based on noteOn's with midinotes rather than frequencies. Early on I considered taking a MIDI sound module approach to norns engine commands, meaning noteOn(note, vel) rather than noteOn(id, note, vel) w/ voice allocation handled sc-side.

doesn't it make sense for things like polyphony and voice stealing logic to stay in the engine where the author knows the performance limitations and character they're after?

Good point. I'm still not sure what's best. Both approaches may be applicable depending on use case - this could be two different engines types. What I find a bit peculiar is the inclusion of id if voice allocation is handled sc-side (the argument for this is it may be needed for alternative tunings and such).

onFree seems to be reliably getting called for me, did you find some cases it isn't?

You're right. My bad. The NodeWatcher is registered in Node.onFree: https://github.com/supercollider/supercollider/blob/develop/SCClassLibrary/Common/Control/Node.sc#L181

markwheeler commented 6 years ago

What I find a bit peculiar is the inclusion of id if voice allocation is handled sc-side (the argument for this is it may be needed for alternative tunings and such).

Yes exactly, it does seem a little redundant but it allows the script to trigger a note of any arbitrary freq, without having to use pitch bend. And then of course the id is required for the noteOff.

We could consider the MIDI sound module approach, I think it would make typical use cases simpler but be less inviting of more experimental ones?

catfact commented 6 years ago

MIDI sound module.... less inviting of more experimental ones?

absolutely

simonvanderveldt commented 6 years ago

not sure if it matches 100% with this topic, but @catfact pointed out if we're going to do some changes to the engines it would be good to unify them as best we could. This in response to some review comments about inconsistencies we currently have https://github.com/monome/dust/pull/171#pullrequestreview-145787148

simonvanderveldt commented 6 years ago

Also, for parameters of (synth) engines would it be useful to look at what other projects (or maybe only projects that build on top of SuperCollider?) are doing? Or do we want to stay close to what SuperCollider offers?

For example: Sonic Pi has quiet a list of things one can pass to a synth, I believe most of it is documented here https://sonic-pi.net/tutorial.html#section-2

catfact commented 6 years ago

i appreciate the desire for common paradigms to be standardized by convention (e.g. polysynth with ADSR amp envelope.)

but i also really want to be really careful about adding proscriptive structure around what sound engines should look like, polyphonic or not

catfact commented 6 years ago

woops didn't mean to close

tehn commented 6 years ago

agreed that it'd be nice to have a class of engines that could be interchangeable, but this should not apply to all engines.

thank you for the thoughts everyone!

lazzarello commented 6 years ago

This issue came to my attention during my FM7 engine project. I could imagine some kind of facility that defines symbols for all known parameters for a polyphonic synthesizer, then a utility that does some kind of collection method to exclude those not implemented by an engine. For example, my FM7 synth doesn't have a filter (and probably won't in the future) so an \lpf, \bpf, etc symbol will never map to a param method.

catfact commented 6 years ago

supercollider gives us the OOP tools to enforce this architecturally.

so something like:

PolyCroneEngine : CroneEngine {
    // 'args' should be... a Dictionary of synth args and values? a special POD class?
    noteOn (id, args) {
        // SC's "pure virtual" : throw error if we try to use this method w/ abstract base class
        this.subclassResponsibility(thisMethod);
    }

    noteOff (id) {
        this.subclassResponsibility(thisMethod);
    }

    // set a given param value for all voices??
    setVoiceParam(name, val) {
        this.subclassResponsibility(thisMethod);
    }
    /// ... etc
}

seems useful for the usual reasons:

inheriting from the interface ensures that you are implementing all the expected stuff
crone/matron can make whatever special decisions for poly engines

issue remains, for me, where exactly to draw the line as far as abstraction. (e.g. no way one realtime "timbre" param is always gonna be enough.) i guess i don't have any strong opinions on this, except for a feeling that limitations will always become onerous at some point, so start by implementing the minimum (note on, note off, stop all notes) and keep things generic when possible (e.g. array of args, since even "note number" or "hz" is kinda insufficient)

if this seems like a good approach then maybe a good exercise would be converting extant poly engines to use a really minimal version of something like this, identifying commonalities and exceptions as they arise

catfact commented 6 years ago

@markwheeler

I don't think the lua script can check if a function exists before calling it.

hm, yeah, i was thinking in terms of required interface methods. but indeed, SC has powerful introspection:

Engine_PolySub.findMethod(\addVoice)

returns -> Engine_PolySub:addVoice (method is implemented)

Engine_PolyPerc.findMethod(\addVoice)

returns -> nil (not implemented)

so, not so much that lua can check, but we explcitly tell lua the whole command/poll interface of the engine at load time, so Crone can check and build appropriate command descriptor table.

markwheeler commented 5 years ago

Is it possible to move this to the norns repo?

neauoire commented 5 years ago

Created a little demo that can look for these standard API methods here. It currently looks for .noteOn, but I've been considering making it generate a table of all engines with their support for each one of these suggested methods.

markwheeler commented 5 years ago

I added an extra variation of noteOn to the top post that I've been using for sample playback engines (Timber).

tehn commented 5 years ago

what's the best way forward with this?

just publish a specification for the docs?

i can certainly update my (pretty boring) engine, and we could encourage migration of those engines that would be valid.

markwheeler commented 5 years ago

Getting an (optional) standard in the docs and applied to the current SC engines sounds like a great first step to me. Then we can discuss further about having a lua class per engine to allow easier checking of an engine's capabilities?

tehn commented 3 years ago

opened in 2018, perhaps this is a pipedream.

unless somebody wants to take this on (speak up!), i'm going to close it.

monome / norns

Standardize SC synth engine commands #844