search

grame-cncm / faust

Functional programming language for signal processing and sound synthesis
http://faust.grame.fr
Other
2.55k stars 321 forks source link

Question: dynamic node (aka live coding or hot reload) systems using faust for their nodes? #685

Open avdrd opened 2 years ago

avdrd commented 2 years ago

I apologize in advance if this is off-topic but I can't seem to find another place to ask.

My understanding is that faust does whole-program optimization and produces a single DSP.

If I want to do the Hot Reloads (or live coding) equivalent for a faust-based system, meaning where the nodes are each a faust DSP, but the extern "linker", or more appropriately bussing system is dynamic and can reload some nodes while keeping others, what options do I have?

Basically, I'm looking for something like Glicol or SuperCollider's Ndefs (JITlib), but with faust nodes. (In a nutshell, Glicol does replacements only on changed AST nodes; it's also rather new and at the proof of concept stage. SC Ndefs are somewhat more established but a bit byzantine, they have an array of slots model, simulating a mixer with some ability to copy from other chains' bus, but also some limitations like occasional dangling bus references, if chains get reallocated.)

I know Csound can load (and reload) faust dsps as instruments, but Csound has been struggling for decades to get a real graph-based node replacement system of their own. They've added a signal flow thing around 2010, but it's still rather static, as discussed here.

So, do you know of anything of a live-coding system based on faust DSPs as nodes?

Also, I asking about textual programming systems. I knew there's faust~ for PD, and I just discovered something called TouchDesigner that can also load faust DSPs, but it doesn't seem too different from PD on a first look, meaning it seem also programmed by drawing graphs on the screen, which is not something that I want.

sletz commented 2 years ago

Some environments can do that either by using the libfaust + LLVM JIT way of the libfaust + WebAssembly way:

avdrd commented 2 years ago

Alas not that useful for what I'm asking:

There's also Pure (not PD), which has faust bindings, but all it seems to be able to do is read data back and forth from the DSP, so not much pre-existing infrastructure there for even static graphs of DSPs.

So, insofar, zero obvious solutions alas for what I'm really asking about... I'll update this later with the ones I'm unclear about.

sletz commented 2 years ago

Faust has a Rust backend. Would it makes sense and technically possible to add Faust support directly in Glicol?

avdrd commented 2 years ago

It seems I'm better off with SOUL for this. It also has spec for patches that seemingly can be JIT compiled and hot reloaded "Patches are JIT-compiled and can be hot-reloaded while running." I'd have to experiment with that. Also, I know that recently there's a Faust backend for SOUL's DSP part.

kmatheussen commented 2 years ago

On Mon, Dec 20, 2021 at 4:46 AM avdrd @.***> wrote:

I apologize in advance if this is off-topic but I can't seem to find another place to ask.

My understanding is that faust does whole-program optimization and produces a single DSP.

If I want to do the Hot Reloads (or live coding) equivalent for a faust-based system, meaning where the nodes are each a faust DSP, but the extern "linker", or more appropriately bussing system is dynamic and can reload some nodes while keeping others, what options do I have?

Basically, I'm looking for something like Glicol https://glicol.org/ or SuperCollider's Ndefs (JITlib), but with faust nodes. (In a nutshell, Glicol does replacements only on changed AST nodes; it's also rather new and at the proof of concept stage. SC Ndefs are somewhat more established but a bit byzantine, they have an array of slots model, simulating a mixer with some ability to copy from other chains' bus, but also some limitations like occasional dangling bus references, if chains get reallocated.)

I know Csound can load (and reload) faust dsps as instruments, but Csound has been struggling for decades to get a real graph-based node replacement system of their own. (They've added a signal flow thing around 2010, but it's still rather static, as discussed here https://csound.com/docs/manual/SignalFlowGraphOpcodes.html.)

So, do you know of anything of a live-coding system based on faust DSPs as nodes?

It's unclear what you mean by a "faust node". If you mean a function in a faust program, that's going to be very difficult. But I guess you mean a faust program? Connecting and editing several simultaneously running faust programs?

There are some options, you can for instance use Pd or supercollider and connect faust modules there, and there is also at least one vst plugin where you can edit faust code in native vst gui.

And there is radium, which I've made: http://users.notam02.no/~kjetism/radium/ . In radium you can do all this inside one program. Audio is updated while you write code and you can connect as many faust programs as you want, and how you want. Both LLVM and the interpreter backends are supported, which gives you the option of whether you want to have immediate response (interpreter) or have a low CPU usage (LLVM). Here is an old video demonstrating programming faust inside radium (GUI is much better now): https://www.youtube.com/watch?v=LJm9Lox1WFA

—

Reply to this email directly, view it on GitHub https://github.com/grame-cncm/faust/issues/685, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIX3J73YJR3VCU24C7BTNDUR2RHPANCNFSM5KMWYJUA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

sletz commented 2 years ago

A SOUL patch basically contains a SOUL graph composed of one or several connected SOUL processors, with a LLVM JIT machinery to do dynamic compilation. This is something you can perfectly do in Faust since a Faust DSP program can describe the same as a SOUL graph (apart some limitations of Faust language in multi-rate graph).

avdrd commented 2 years ago

If you mean a function in a faust program, that's going to be very difficult.

Well, yeah, I know, because of whole-program optimization. Something like SC or Glicol that is "ugen-based" makes that replacement bit fairly easy but loses whole-program optimization. It's somewhat more nicely done in Glicol as it inspects what you changed exactly in the AST, e.g. if you just change the frequency, the oscillator node will keep running and maintain its phase.

By the way, I was under the impression that SOUL is fully open source, because it's on github, but they've only open sourced the top-half of it, that produces the IR, not the sound server that plays it. Although with their vision (see their ADC 2018 talk) that the latter belongs in hardware, I guess that's somewhat understandable... but not completely. After all there were open source releases of the software OpenGL stack, Mesa etc., if we take their hardware analogy. So, I'm a bit reluctant to jump on the SOUL board for that reason.

I'll have a look at Radium for the hot reloading part, thanks, but I wasn't really looking for the DAW approach.

sletz commented 2 years ago

The SOUL project is currently in sleep mode after ROLI bankruptcy. It may resurrect but nothing has been announced yet.

avdrd commented 2 years ago

@sletz I had no idea. Thanks for letting me know.

sletz commented 2 years ago

Note that part of the Faust compiler machinery can be used even if you decide not to use the Faust programming language itself. See https://faustdoc.grame.fr/tutorials/signal-api/.

sletz commented 2 years ago

@avdrd see https://github.com/chaosprint/glicol/issues/25 ?

avdrd commented 2 years ago

Well, I saw it... but that doesn't help me much.

As for SOUL, the patch hot-loading thing alas turned out to be a red herring. I misunderstood its granularity. The actual graphs are defined with graph inside the .soul files. And it looks like there's no partial or hot reloading of those, i.e. the whole graph with all processor nodes gets reloaded.

chaosprint commented 2 years ago

@avdrd I am wondering what your use case would be and what features you wish to see on Glicol next?

avdrd commented 2 years ago

@chaosprint There's a space in live coding for changing the DSP on the fly as well, not just selecting from a library of ugens. In Extempore it works well enough for small DSPs, meaning the JIT compile time isn't a drag. But there's all that awesome s-expression language to deal with. And flakiness on non-Macs with Extempore in general.

Faust being based on a functional-programming and having pretty compact syntax makes it an interesting choice for such. Also, one can even transpose the syntax into some other language, as long as the paradigm is close enough as noted above https://github.com/grame-cncm/faust/issues/685#issuecomment-997841509 So that would be pretty interesting project if not done already.

They have yet to get a Faust JIT working for SuperCollider, by the way. To load new Fasust DSPs, the SC server needs to shut down and restarted. Not exactly live performance in that regard.

sletz commented 2 years ago

They have yet to get a Faust JIT working for SuperCollider, by the way. To load new Faust DSPs, the SC server needs to shut down and restarted. Not exactly live performance in that regard.

WIP here : https://github.com/madskjeldgaard/faustgen-supercollider