RustAudio / dsp-chain

A library for chaining together multiple audio dsp processors/generators, written in Rust!
MIT License
298 stars 20 forks source link

Other user-friendly API ideas. #49

Open mitchmindtree opened 9 years ago

mitchmindtree commented 9 years ago

Currently the trait-based node graph approach is very fast, efficient, and quite user friendly. However, it has the tendency to lead toward a slightly more OOP design, which can often feel like a bit of an up-hill battle with Rust.

I'm interested in coming up with some ideas for a more user-friendly, perhaps functional-esque way of describing the node graph.

One idea I was considering was to implement the bitshift operators for describing a series of inputs in a sort of Chain. I.e.

let chain: Chain = output << master << bus << effect << effect << synth;

where each element in the chain can be either

where T: Node.

I'll keep thinking on this and try come up with some better examples.

bvssvni commented 9 years ago

I like this idea.

errordeveloper commented 9 years ago

:+1:

mitchmindtree commented 9 years ago

Ok, so one issue that I'm finding hard to avoid with this is approach is performance.

Most DSP types will have to mutate/update some internal state (i.e. phase, ringbuffer, etc) during the audio loop. The simplest way of handling this that I've been able to come up with so far is to simply make the Node and Dsp traits' audio_requested method take a mutable reference to self (as is currently the case) so that a user can update the type's state at the same time audio is requested. I'm not sure it is possible to decouple these things in a way that is as performant, as for many types their mutation is dependent upon the results of their children's audio_requested methods.

Rust's bit-shift operator takes self as an immutable reference, meaning we'd have to at some point separate the immutable state from the mutable state. As mentioned above, I'm not sure if there is a solution to this that is practical, ergonomic or even possible while retaining the same performance (I'm open to ideas though).

However, since this issue was originally posted, a Graph type has been added to the crate which behaves very similarly to the Chain type described above (without the fancy bit-shift operator). The Graph type and Dsp trait are designed for use together, where the Graph can handle any type that implements the Dsp trait (this includes Graph itself). The most special property of the Graph type is that it allows for multiple inputs and outputs (whereas the Node trait only allows multiple inputs), safely solving the problem of dealing with multiple mutable references and ownership.

The Node trait still remains, however it is unrelated to the Graph and Dsp traits and is designed for a different use case. The following two paragraphs (from the README) describe the difference between use cases for the Graph type and the Node trait.

The Node trait offers a DSP chaining design via its inputs method. It is slightly simpler to use than the Graph type however also slightly more limited. Using the Node trait, it is impossible for two nodes to reference the same input Node making it difficult to perform tasks like complex "bussing" and "side-chaining".

The Graph type constructs a directed, acyclic graph of DSP nodes. It is the recommended approach for more advanced DSP chains that involve things like "bussing", "side-chaining" or more DAW-esque behaviour. The Graph type requires its nodes to have implemented the Dsp trait (a slightly simplified version of the Node trait, though entirely unrelated). Internally, Graph uses bluss's petgraph crate.

Here's the old Node example and here's the new Graph example if you're interested.

Another area for API design that I think is definitely worth exploring is rust's awesome Iterator trait. For example, perhaps rather than passing a mutable buffer into an .audio_requested method, perhaps each Dsp type could return some Iterator<Item=Sample> that would generate a buffer worth of samples. This may offer some performance advantages by reducing the number of Vec allocations necessary... but we'd have to do some experimentation first. It's also likely that the Iterator may need to store a mutable reference to self, possibly causing some ownership issues, but i'm sure they could be dealt with safely at least within the Graph type. Also, in rust's current state I imagine this would be considerably trickier to implement than audio_requested currently is due to the difficulty of specifying Iterator return types. This would however be solved by something like abstract/anonymous return types which seems to be very high priority for post-1.0 - perhaps it is best to wait until then for further experimentation on this.

Finally, I'd like to mention that the more I use Rust, the more I feel it is impractical to try and opt for either a purely OOP or a purely FP approach in terms of design. If anything, the style would probably be more accurately be described as "Ownership Oriented Programming" which dips into the benefits of both of the previously mentioned paradigms:

Edit: It seems the ops::{shl,shr} have changed since last time I checked and now take self by value, so maybe the idea's back on the table.

errordeveloper commented 9 years ago

My take would be to go with build-time approach with a syntax extension of this kind of shape:

let chain: Chain = chain!(
    mixer: params,
    channels: {
      nameA: [effectA1 << effectA2 << synthA],
      nameB: [effectB1 << effectB2 << synthB],
      nameC: [effectC1 << mic_input],
      nameD: subChainD,
    }
);
bvssvni commented 9 years ago

One idea is using declarative syntax and then feed it to a "black box" that is free to optimize or represent it other ways.

mitchmindtree commented 8 years ago

@errordeveloper I just remembered your comments about taking a compile-time approach to signal graph building - you might be interested in the latest changes to the RustAudio/sample - in particular, the Signal trait. Here's an example of it being used to build a very basic synth. I think mostly it will render dsp-chain much less useful, as it achieves most of the same properties which a much more "fundamentals-oriented" simplicity. However, I think dsp-chain might still be more suited to highly dynamic graphs. I'll try to get around to doing a comparison table of the two soon (for myself as much as anyone else heh!).

Edit: oh yeah, I forgot to mention that perhaps the sample crate might be a more suitable target for the macro idea you proposed above.?