Sockets server to receive data from a python program

[rabid-inventor]

Hi Sam It's Gee from pimoroni :) You Probably get this all the time but assuming that I can write a ruby socket server into the sonic pi window. is it possible at the moment to have python send data from gpio devices into variables in sonic pi. :)

thinking of something like this http://www.tutorialspoint.com/ruby/ruby_socket_programming.htm

I have done something similar with pure data just want to get your opinion on it

[josephwilk]

Just a warning @rabid-inventor that the exact protocol used by SonicPi is currently under development. Its unlikely to stay as UDP.

[rabid-inventor]

Thanks for the heads up sockets for the moment seems to be accepting data so i'll see how it goes :)

[developius]

Could it go the other way? Have Sonic Pi send data (current note or sound) through a socket to Python?

[josephwilk]

sonic-pi uses OSC and the server sends and receives osc messages. So yes if you listen to the right port sonic-pi will send data. Though you wave goodbye to the sonic-pi server which does all the magic with supercolider and sound.

[developius]

@josephwilk can I actually get the sound data right from a socket that Sonic Pi uses (if it does?)? E.g. play a note and grab the sound data that Sonic Pi is using in it's server.

[samaaron]

Hey @developius, what do you mean by the 'sound data'? Sonic Pi's Ruby server sends OSC messages about which synths to play and when - including which FX to set up, busses, groups etc. All low-level scsynth management stuff.

[samaaron]

Hey @rabid-inventor how did you experiments with Sonic Pi's socket comms go?

[developius]

@samaaron I have no idea about this audio stuff. Ideally want I would like is to be able to get Sonic Pi to transmit the OSC data through a socket to a waiting Python program which can then deal with it.

[samaaron]

Well, the audio stuff is all handled by SuperCollider. Sonic Pi's Ruby server essentially sends control and manipulation info. Here's a raw stream of the kind of info that's sent:

Here's a brief excerpt of

[2015-01-27 16:54:29] Starting server thread
[2015-01-27 16:54:29] [2015-01-27 16:54:29] [2015-01-27 16:54:29] Booting Sonic Pi
[2015-01-27 16:54:29] ----------------
[2015-01-27 16:54:29] [2015-01-27 16:54:29] Booting on OS X
[2015-01-27 16:54:29] Starting the SuperCollider server...
[2015-01-27 16:54:29] System: '/Users/sam/Development/RPi/sonic-pi/app/server/native/osx/scsynth' -a 1024 -u 4556 -m 131072&
[2015-01-27 16:54:29] SRV info        - Initialising comms... #<Thread::Queue:0x007f7f93482f00>
[2015-01-27 16:54:29] SRV info        - Clearing scsynth
[2015-01-27 16:54:29] SRV grp f 00000 - Clear 0
[2015-01-27 16:54:29] OSC             ~ /clearSched []
[2015-01-27 16:54:29] SRV info        - Requesting notifications
[2015-01-27 16:54:29] SRV info        - Loading synthdefs from path: /Users/sam/Development/RPi/sonic-pi/etc/synthdefs
[2015-01-27 16:54:29] OSC             ~ /g_freeAll [0]
[2015-01-27 16:54:29] OSC             ~ /notify [1]
[2015-01-27 16:54:29] OSC             ~ /d_loadDir ["/Users/sam/Development/RPi/sonic-pi/etc/synthdefs"]
[2015-01-27 16:54:29] OSC             ~ /sync [1]
[2015-01-27 16:54:29] SRV info        - Clearing scsynth
[2015-01-27 16:54:29] OSC             ~ /clearSched []
[2015-01-27 16:54:29] SRV grp f 00000 - Clear 0
[2015-01-27 16:54:29] SRV grp n 00002 - Create [STUDIO-MIXER:2] head 0
[2015-01-27 16:54:29] OSC             ~ /g_freeAll [0]
[2015-01-27 16:54:29] OSC             ~ /g_new [2, 0, 0]
[2015-01-27 16:54:30] OSC             ~ /g_new [3, 2, 2]
[2015-01-27 16:54:30] SRV grp n 00003 - Create [STUDIO-FX:3] before [STUDIO-MIXER:2]
[2015-01-27 16:54:30] SRV grp n 00004 - Create [STUDIO-SYNTHS:4] before [STUDIO-FX:3]
[2015-01-27 16:54:30] OSC             ~ /g_new [4, 2, 3]
[2015-01-27 16:54:30] SRV grp n 00005 - Create [STUDIO-RECORDING:5] after [STUDIO-MIXER:2]
[2015-01-27 16:54:30] OSC             ~ /g_new [5, 3, 2]
[2015-01-27 16:54:30] SRV nde t 00006 - Trigger <sonic-pi-mixer:6> head [STUDIO-MIXER:2]
[2015-01-27 16:54:30] BDL    -1.00000 ~ [-1:1422377670] /s_new ["sonic-pi-mixer", 6, 0, 2, "in_bus", 10]
[2015-01-27 16:54:31] SRV nde c 00006 - Control <6> with args: {"invert_stereo"=>0}
[2015-01-27 16:54:31] OSC             ~ /n_set [6, "invert_stereo", 0.0]
[2015-01-27 16:54:31] SRV nde c 00006 - Control <6> with args: {"force_mono"=>0}
[2015-01-27 16:54:31] OSC             ~ /n_set [6, "force_mono", 0.0]
[2015-01-27 21:17:12] SRV nde t 00008 - Trigger <sonic-pi-basic_mixer:8> head [STUDIO-MIXER:2]
[2015-01-27 21:17:12] SRV grp n 00007 - Create [Run-1-Synths:7] tail [STUDIO-SYNTHS:4]
[2015-01-27 21:17:12] SRV grp n 00009 - Create [Run-1-FX:9] tail [STUDIO-FX:3]
[2015-01-27 21:17:12] OSC             ~ /g_new [7, 1, 4]
[2015-01-27 21:17:12] OSC             ~ /s_new ["sonic-pi-basic_mixer", 8, 0, 2, :amp, 1, :amp_slide, 0.1, :amp_slide_shape, 5, :amp_slide_curve, 0, "in_bus", 12, "amp", 0.3,
                                         "out_bus", 10]
[2015-01-27 21:17:12] OSC             ~ /g_new [9, 1, 3]
[2015-01-27 21:17:12] OSC             ~ /b_allocRead [0, "/Users/sam/Development/RPi/sonic-pi/etc/samples/bd_haus.wav", 0, 0]
[2015-01-27 21:17:12] OSC             ~ /g_new [10, 0, 9]
[2015-01-27 21:17:12] SRV grp n 00010 - Create [Run-1-reverb:10] head [Run-1-FX:9]
[2015-01-27 21:17:12] SRV nde t 00011 - Trigger <sonic-pi-zawa:11> head [Run-1-Synths:7]
[2015-01-27 21:17:12] BDL     0.00000 ~ [0.0:1422393432] /s_new ["sonic-pi-zawa", 11, 0, 7, :note, 36, :note_slide, 0, :note_slide_shape, 5, :note_slide_curve, 0, :amp, 1, :a
                                        mp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curve, 0, :attack, 0, :
                                        decay, 0, :sustain, 0, :release, 5, :attack_level, 1, :sustain_level, 1, :cutoff, 60, :cutoff_slide, 0, :cutoff_slide_shape, 5, :cutof
                                        f_slide_curve, 0, :res, 0.1, :res_slide, 0, :res_slide_shape, 5, :res_slide_curve, 0, :phase, 0.25, :phase_slide, 0, :phase_slide_shap
                                        e, 5, :phase_slide_curve, 0, :phase_offset, 0, :wave, 1, :invert_wave, 1, :range, 24, :range_slide, 0, :range_slide_shape, 5, :range_s
                                        lide_curve, 0, :disable_wave, 0, :pulse_width, 0.5, :pulse_width_slide, 0, :pulse_width_slide_shape, 5, :pulse_width_slide_curve, 0, "
                                        out_bus", 12.0]
[2015-01-27 21:17:12] SRV nde t 00012 - Trigger <sonic-pi-fx_reverb:12> head [Run-1-reverb:10]
[2015-01-27 21:17:12] SRV nde t 00013 - Trigger <sonic-pi-dsaw:13> head [Run-1-Synths:7]
[2015-01-27 21:17:12] BDL     0.00000 ~ [0.0:1422393432] /s_new ["sonic-pi-fx_reverb", 12, 0, 10, :amp, 1, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :mix, 0.4,
                                         :mix_slide, 0, :mix_slide_shape, 5, :mix_slide_curve, 0, :pre_amp, 1, :pre_amp_slide, 0, :pre_amp_slide_shape, 5, :pre_amp_slide_curv
                                        e, 0, :room, 1, :room_slide, 0, :room_slide_shape, 5, :room_slide_curve, 0, :damp, 0.5, :damp_slide, 0, :damp_slide_shape, 5, :damp_sl
                                        ide_curve, 0, "in_bus", 14.0, "out_bus", 12.0]
[2015-01-27 21:17:12] BDL     0.00000 ~ [0.0:1422393432] /s_new ["sonic-pi-dsaw", 13, 0, 7, :note, 62, :note_slide, 0, :note_slide_shape, 5, :note_slide_curve, 0, :amp, 1.180
                                        931950894791, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curve, 
                                        0, :attack, 0, :decay, 0, :sustain, 0, :release, 0.1, :attack_level, 1, :sustain_level, 1, :env_curve, 2, :cutoff, 86.54046293953087, 
                                        :cutoff_slide, 0, :cutoff_slide_shape, 5, :cutoff_slide_curve, 0, :detune, 12, :detune_slide, 0, :detune_slide_shape, 5, :detune_slide
                                        _curve, 0, "out_bus", 14.0]
[2015-01-27 21:17:12] OSC             ~ /sync [2]
[2015-01-27 21:17:12] OSC             ~ /b_query [0]
[2015-01-27 21:17:12] SRV nde t 00014 - Trigger <sonic-pi-basic_stereo_player:14> head [Run-1-Synths:7]
[2015-01-27 21:17:12] BDL     0.00000 ~ [0.0:1422393432] /s_new ["sonic-pi-basic_stereo_player", 14, 0, 7, :amp, 2.5, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0,
                                         :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curve, 0, :rate, 1, :rate_slide, 0, :rate_slide_shape, 5, :rate_slide_curve, 
                                        0, :buf, 0, "out_bus", 12.0]
[2015-01-27 21:17:12] SRV nde t 00015 - Trigger <sonic-pi-dsaw:15> head [Run-1-Synths:7]
[2015-01-27 21:17:12] BDL     0.12500 ~ [0.125:1422393432] /s_new ["sonic-pi-dsaw", 15, 0, 7, :note, 40, :note_slide, 0, :note_slide_shape, 5, :note_slide_curve, 0, :amp, 0.6
                                        483619696810659, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curv
                                        e, 0, :attack, 0, :decay, 0, :sustain, 0, :release, 0.1, :attack_level, 1, :sustain_level, 1, :env_curve, 2, :cutoff, 76.8793080635116
                                        , :cutoff_slide, 0, :cutoff_slide_shape, 5, :cutoff_slide_curve, 0, :detune, 12, :detune_slide, 0, :detune_slide_shape, 5, :detune_sli
                                        de_curve, 0, "out_bus", 14.0]
[2015-01-27 21:17:12] SRV nde t 00016 - Trigger <sonic-pi-dsaw:16> head [Run-1-Synths:7]
[2015-01-27 21:17:12] OSC             ~ /b_allocRead [1, "/Users/sam/Development/RPi/sonic-pi/etc/samples/guit_em9.wav", 0, 0]
[2015-01-27 21:17:12] BDL     0.25000 ~ [0.25:1422393432] /s_new ["sonic-pi-dsaw", 16, 0, 7, :note, 64, :note_slide, 0, :note_slide_shape, 5, :note_slide_curve, 0, :amp, 0.98
                                        66607781688543, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curve
                                        , 0, :attack, 0, :decay, 0, :sustain, 0, :release, 0.1, :attack_level, 1, :sustain_level, 1, :env_curve, 2, :cutoff, 92.72450887841762
                                        , :cutoff_slide, 0, :cutoff_slide_shape, 5, :cutoff_slide_curve, 0, :detune, 12, :detune_slide, 0, :detune_slide_shape, 5, :detune_sli
                                        de_curve, 0, "out_bus", 14.0]
[2015-01-27 21:17:12] OSC             ~ /sync [3]
[2015-01-27 21:17:12] OSC             ~ /b_query [1]
[2015-01-27 21:17:12] SRV nde t 00017 - Trigger <sonic-pi-stereo_player:17> head [Run-1-Synths:7]
[2015-01-27 21:17:12] BDL     0.25000 ~ [0.25:1422393432] /s_new ["sonic-pi-stereo_player", 17, 0, 7, :amp, 1, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 
                                        0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curve, 0, :attack, 0, :decay, 0, :sustain, -1, :release, 0, :attack_level, 1, :susta
                                        in_level, 1, :env_curve, 2, :rate, -1, :start, 0, :finish, 1, :buf, 1, "out_bus", 12.0]
[2015-01-27 21:17:12] SRV nde t 00018 - Trigger <sonic-pi-dsaw:18> head [Run-1-Synths:7]
[2015-01-27 21:17:12] BDL     0.37500 ~ [0.375:1422393432] /s_new ["sonic-pi-dsaw", 18, 0, 7, :note, 62, :note_slide, 0, :note_slide_shape, 5, :note_slide_curve, 0, :amp, 0.5
                                        693577413457428, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curv
                                        e, 0, :attack, 0, :decay, 0, :sustain, 0, :release, 0.1, :attack_level, 1, :sustain_level, 1, :env_curve, 2, :cutoff, 103.706013646039
                                        38, :cutoff_slide, 0, :cutoff_slide_shape, 5, :cutoff_slide_curve, 0, :detune, 12, :detune_slide, 0, :detune_slide_shape, 5, :detune_s
                                        lide_curve, 0, "out_bus", 14.0]
[2015-01-27 21:17:12] SRV nde t 00020 - Trigger <sonic-pi-basic_stereo_player:20> head [Run-1-Synths:7]
[2015-01-27 21:17:12] SRV nde t 00019 - Trigger <sonic-pi-dsaw:19> head [Run-1-Synths:7]
[2015-01-27 21:17:12] BDL     0.50000 ~ [0.5:1422393433] /s_new ["sonic-pi-basic_stereo_player", 20, 0, 7, :amp, 2.5, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0,
                                         :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curve, 0, :rate, 1, :rate_slide, 0, :rate_slide_shape, 5, :rate_slide_curve, 
                                        0, :buf, 0, "out_bus", 12.0]
[2015-01-27 21:17:12] BDL     0.50000 ~ [0.5:1422393433] /s_new ["sonic-pi-dsaw", 19, 0, 7, :note, 69, :note_slide, 0, :note_slide_shape, 5, :note_slide_curve, 0, :amp, 1.032
                                        1752743464754, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curve,
                                         0, :attack, 0, :decay, 0, :sustain, 0, :release, 0.1, :attack_level, 1, :sustain_level, 1, :env_curve, 2, :cutoff, 98.33295766480863,
                                         :cutoff_slide, 0, :cutoff_slide_shape, 5, :cutoff_slide_curve, 0, :detune, 12, :detune_slide, 0, :detune_slide_shape, 5, :detune_slid
                                        e_curve, 0, "out_bus", 14.0]
[2015-01-27 21:17:13] SRV nde t 00021 - Trigger <sonic-pi-dsaw:21> head [Run-1-Synths:7]
[2015-01-27 21:17:13] BDL     0.62500 ~ [0.625:1422393433] /s_new ["sonic-pi-dsaw", 21, 0, 7, :note, 55, :note_slide, 0, :note_slide_shape, 5, :note_slide_curve, 0, :amp, 1.0
                                        655972240844842, :amp_slide, 0, :amp_slide_shape, 5, :amp_slide_curve, 0, :pan, 0, :pan_slide, 0, :pan_slide_shape, 5, :pan_slide_curv
                                        e, 0, :attack, 0, :decay, 0, :sustain, 0, :release, 0.1, :attack_level, 1, :sustain_level, 1, :env_curve, 2, :cutoff, 92.1056881055527
                                        4, :cutoff_slide, 0, :cutoff_slide_shape, 5, :cutoff_slide_curve, 0, :detune, 12, :detune_slide, 0, :detune_slide_shape, 5, :detune_sl
                                        ide_curve, 0, "out_bus", 14.0]

[developius]

Aha, so the last entry in that extract, where it says :note, 50 is the note that's getting played? Would it be possible to 'tap' into this control and manipulation stream and then pull out the note - is it communicating through a socket?

[samaaron]

Yep, that would be possible - although it might be easier to monkey patch the synth triggering fns such as trigger_inst (https://github.com/samaaron/sonic-pi/blob/master/app/server/sonicpi/lib/sonicpi/mods/sound.rb#L2419). However, it's important to point out that fns like this are not part of the public API and are subject to change at any time.

[developius]

Could you tell me which port the stream above is transmitted through - it isn't 4558 by any chance? Also, do you have any docs for the public API or is it being released in V3.0? Sorry to keep bugging you but I'm really interested and think Sonic Pi is a wonderful tool!

[samaaron]

By default scsynth is started on port 4557 which is where all these OSC message are sent. Also, the network API isn't public yet - so it's important to point out that writing code at this low level isn't supported and future versions of Sonic Pi may break your code.

[developius]

Thanks very much - I'll bear that in mind!

[samaaron]

It sends it from a local port - scsynth sends replies back to the 'from' field in the UDP packet.

[developius]

Sorry - typo, I meant if Sonic Pi sends to scsynth:4557, what port does scsynth send back from?

[samaaron]

To whatever port the UDP packet originated - which isn't known until its created.

[developius]

Would it be possible to be a 'man in the middle'? Sonic-Pi -> Me (copy OSC) -> scsynth?

[jweather]

Yes, this is possible. The easiest thing would be to change the port number passed to SC in scsynthexternal.rb. I'm curious what you're interested in doing with the OSC stream.

Depending on what you're trying to do, you could also redefine functions on the Ruby side to get the information you want.

[developius]

@jweather I'm thinking about doing a Sonic Pi visualiser with @musicboyben (http://github.com/musicboyben/SonicVisualiser) that could tap into either the Sonic Pi OSC stream or directly into (any) computer's audio output (I have no idea about that though!) The code in the above repo is just testing really, I'm not even sure the audio capture script even works (although I seem to get output). Would be glad of any pointers!

[samaaron]

This is now possible since v3.0 by sending OSC messages to Sonic Pi over UDP on port 4559 :-)

[developius]

Impressive issue diligence @samaaron - three years! Thanks for the heads up, awesome to hear.