search

shorepine / amy

AMY - A high-performance fixed-point Music synthesizer librarY for microcontrollers
https://shorepine.github.io/amy/
MIT License
184 stars 11 forks source link

First try at pushing "mutable" PCM patches. Here's my code #106

Closed octetta closed 2 weeks ago

octetta commented 6 months ago

can allow pcm_map entries to be added at run-time. Enabled via -DPCM_MUTABLE in Makefile. Usage example in amy-message.c.

bwhitman commented 6 months ago

Neat. It's definitely good for users to be able to play back their own PCM samples. It's something we want to add.

The way this is implemented can't be merged as it would violate our (unwritten, sorry, I'll get to that) law that any control of AMY can happen over the wire protocol. The way this PR Is setup, your feature would only work for people that can write their own front ends to AMY. It also would mean none of AMY would work for any MCU as keeping pcm_samples out of const would put it in RAM. None of our target MCUs could support that.

A middle ground would be to support an event containing a pointer to a pcm sample. Keep the existing samples in const, but allow for pcm samples starting at PCM_PATCHES to be set by sending pointers and lengths (and the looping metadata, sample rate, etc) across as something like event.pcm_pointer. This still violates our (unwritten) law as this couldn't work over the wire protocol on remote systems or across address boundaries. But it would work for people directly including AMY in their software -- like Tulip, Arduino and desktop coders.

Another solution is to send the PCM over the wire protocol as (e.g.) base64 encoded strings. There's some limits we'd hit on message length. I could imagine us inventing a "continuation" character like SYSEX has so that we can send over multiple messages within a smaller buffer size. This sounds messy and I'm scared to imagine how this would work on e.g. Alles, but would resolve the wire protocol issue. You can see partials.py to see how we chunk up and send thousands of AMY messages a second to "stream" data.

Overall, unless we hit upon some magic solution for this i think adding custom PCM samples at runtime is a very "library on top of AMY" solution, not one we need to have as part of AMY itself. For example, for Tulip, we're trying to figure out how to mix PCM samples (from e.g. mp3 decoded playback on Tulip) with AMY. Our solution there will likely not be to adapt AMY directly to support arbitrary PCM, but to grab a function pointer / callback from AMY's audio renderer and mix in the PCM samples at playback time ourselves.

(Another maybe fun solution for you, if i remember your use case right -- you can dynamically compile AMY inline within your front-end code, right? You could just adapt pcm_X.h in your front end's runtime and do a quick compile and dynamic reload?)

octetta commented 6 months ago

Thanks for the feedback! This gives me a way to think about this that helps my use-case as well, as I'm a strong believer in wire-protocol. Last year, I tinkered around with JSON when using Elixir to exchange waveforms with AMY, but I'm not a fan of JSON parsing. Let me tinker with this for a bit... maybe I'll write my thoughts down as a plan before writing a bunch of code this time. Any thoughts on the "per patch sample rate" being part of pcm_map_t though?

bwhitman commented 2 weeks ago

I'm going to close this for now. If you haven't built this yet, make sure to try our memorypcm.c that we ship with Tulip. It loads as an AMY custom oscillator, so can be compiled right in and does mutable PCM patches. https://github.com/shorepine/tulipcc/blob/main/tulip/shared/memorypcm.c