search

josephwilk / shaderview

Light show for live coding
MIT License
78 stars 1 forks source link

Any chance to get live music data into shaderview? #6

Closed mattrei closed 8 years ago

mattrei commented 9 years ago

Hi Joseph,

do you know of any way to get live synth data, like the volume, the waveform, of the programmed sound into shaderview? Similar like it was possible in shadertone?

Thanks, Matthias

josephwilk commented 9 years ago

Yes, its definitely possible, but not tried it out yet.

In effect all Overtone is doing is registering the a message send (using send-reply) and then creating a listener which will update an atom

https://github.com/overtone/overtone/blob/8b8353b27c755004dc0733857bcfee9ce15c6b73/src/overtone/sc/cgens/tap.clj#L10-L33

And

https://github.com/overtone/overtone/blob/8b8353b27c755004dc0733857bcfee9ce15c6b73/src/overtone/sc/synth.clj#L541

 (on-event "/overtone/tap" #'update-tap-data ::handle-incoming-tap-data)

It would require you to compile the synths yourself with the added tap.

Once you have the tap setup + the listener on the tap send-message its a case of rather than setting up an atom, send an OSC message to Shaderview. There might be a more direct route but thats the best thing that comes to mind.

mattrei commented 9 years ago

Hi Joseph, thanks for your detailed answer.

I want to use SonicPi for that and as you know the synths are precompiled there.

I was now looking into a different area to solve the problem. One can use the Icecast streaming server that captures the music from SonicPi and streams it to a client, like ShaderView or whatever Viz tool. Of course there is no support yet from ShaderView - there needs to be support for that. See this tutorial here on how to configure Overtone (works the same for SonicPi - tested): https://groups.google.com/forum/#!topic/overtone/k6UYtxgHb8w

I am doing my visuals on the browser (not with ShaderView) so for me thats a good solution (probably) to go. I will publish my work when its ready.

Thanks for your inspirational work!!!

Best

josephwilk commented 9 years ago

Ohh icecast sounds interesting. Thanks for the pointer. Look forward to seeing how it goes. The browser is a very compelling path, just hard to reconcile with a raspberry pi where browsers suck cpu. Will look forward to hearing your experiences!

mattrei commented 8 years ago

hmm, seems that web streaming is not built for low latency. I run it all localhost: capture the audio from sonic-pi with "darkice" that sends it forth to "icecast2" which then streams it to my browser. Sounds good in theory, however I get a latency with some secs. Thats not usable really for visualization :-1: See also http://stackoverflow.com/questions/17133689/icecast-and-darkice-combination-for-live-streaming

Maybe there exists some different technology. Will also ask on sonic-pi's Issues forum