Sensor data input

[icubex]

We're interested in adding some code so as to allow easy use of sensor data in Sonic Pi. We'd first want to integrate our PiShield (see https://infusionsystems.com/pishield), which uses the Raspberry Pi's SPI and I2C buses. The PiShield can be used with any analog or digital (I2C) sensor, not just I-CubeX. The code to get the sensor data is pretty simple but we need some advice how to make this data available in the Sonic Pi environment. How should we go about this ? Would you have some example or guidelines from which we could work ?

[samaaron]

Hi there, the easiest way to get data into Sonic Pi is to send OSC messages to port 4559. Most programming languages have an OSC library as it's a pretty basic protocol (http://opensoundcontrol.org).

Once you can send OSC messages, Sonic Pi receives and displayed them automatically on screen, so it's super easy to debug. Once you see the messages on screen it's just the case of calling sync to wait for the next message to do stuff. For more information from Sonic Pi's perspective please take a look at Section 12 of the tutorial: http://sonic-pi.net/tutorial.html#section-12

[icubex]

We've already done some tests on a RPi 3B sending OSC msgs at 50 Hz with a very simple Python 3 script using python-osc and the latency was too high to be useful for live performance. Wouldn't the latency be lower if the PiShield sampling would be done within Sonic Pi ?

[samaaron]

Did you enable soft-real-time mode with use_realtime?

Also, are you observing latency in the received OSC messages appearing in the cue log or the audio triggered? If it’s the latter, this is a known problem on the pi which does suffer from large audio buffers introducing audio latency. This can be tuned somewhat with some jack shenanigans.

[icubex]

The latency is apparent by the delay it takes before the audio is triggered following a tap on a Touch (FSR) sensor. The sound is output with a fe-pi v2 hat.

This is the Sonic Pi code: ` triggered = false

live_loop :foo do use_real_time s0, s1, s2 = sync "/osc/ICubeX/analog" if s0 > 100 and not triggered a = 64 b = s1 127 / 1023 c = s2 127 / 1023 synth :prophet, note: a, cutoff: b, sustain: c triggered = true else if s0 < 10 triggered = false end end end `

This is the Python code: ` import spidev import time import argparse from pythonosc import osc_message_builder from pythonosc import udp_client import sys

PORT=4559 MSG_ADDR='/ICubeX/analog'

NUM_CH = 8 adcValues = [0 for i in range(NUM_CH)]

spi = spidev.SpiDev() #init SPI device spi.open(0,0) #open SPI port 0 spi.max_speed_hz = 1000000 # required for Raspbian Stretch

def readADC(ch): if ( (ch>NUM_CH-1) or (ch<0) ): return -1 r= spi.xfer2([1,(8+ch)<<4,0]) val = ((r[1]&3)<<8) + r[2]
return val

if name == "main": parser = argparse.ArgumentParser() parser.add_argument("--ip", default="127.0.0.1", help="The ip of the OSC server") parser.add_argument("--port", type=int, default=PORT, help="The port the OSC server is listening on") args = parser.parse_args()

client = udp_client.SimpleUDPClient(args.ip, args.port)

while 1: try: time.sleep(0.02) #50 hz output for ch in range(0, NUM_CH): val = readADC(ch) adcValues[ch] = readADC(ch) print("ADC Values = ", adcValues) client.send_message(MSG_ADDR, adcValues) except KeyboardInterrupt: break

print("\n\ngoodbye.") `

[icubex]

FYI, we have also experimented with Pyo (http://ajaxsoundstudio.com/software/pyo) and found negligible latency.

[rbnpi]

I have done quite a few projects interfacing Sonic Pi to various sensors, using python-osc as the link. The nearest to what you are trying is probably this one https://rbnrpi.wordpress.com/new-sonic-pi-theremin-using-time-of-flight-laser-sensor/ As Sam says audio latency is quite a big problem is you are trying to do real time musical input. Running Sonic Pi on a Mac gives much better results. By way of experiment try sending the OSC message from the PI running your interface to SP running on a machine with much lower latency. I find that works quite well: even better if one (or both) machines have wired connections to your local network. This may be a worthwhile experiment to try. Recently I have done work on using broadcast messages from a python script to various machines running Sonic Pi and I manage to get good results, especially correcting for audio latency differences between different types of computer running Sonic Pi. https://in-thread.sonic-pi.net/t/sonic-pi-orchestra/1614

[icubex]

@samaaron Perhaps our question should be rephrased. The OSC route is possibly easiest but not the most elegant nor user friendly and most likely not the fastest or most efficient. So we'd like to know where in the Sonic Pi code we could add the SPI code that assigns the ADC values to global variables (that could then be used in Sonic Pi user code) and add a UI element in the prefs to turn on/off the PiShield.

[samaaron]

Hi @icubex, I'm not sure how OSC isn't user friendly as it essentially requires zero config and code from a Sonic Pi user's perspective to receive and monitor. Using the local loopback network will mean that incredibly high Hz isn't feasible, but anything above 20hz is likely to not be very useful from within a Sonic Pi live loop anyway as these messages in turn are sent to SuperCollider (the underlying synth engine also through OSC messages). Essentially whatever route you use to send Sonic Pi data, it will always end up as a local loopback OSC message before it reaches the synth engine.

For the record, Sonic Pi does not support global variables. These are non-deterministic and non-recordable - both things that Sonic Pi eschews. All state change goes through our immutable, totally-ordered time-state event system which allows for recording state changes automatically (again with zero code).

The delay you're hearing is very likely to be in the audio chain, most likely within the jack audio routing server. This is noticeable even when simply routing the audio in to the audio out - not hitting OSC or Sonic Pi's language runtime. For a test, you might want to ping back an OSC message received to your Python script and time that. You should find it's just a few milliseconds.

Unfortunately the prefs UI is not end-user customisable at this stage. We have very little funds to continue development and can only focus on features that benefit all platforms which includes macOS and Windows.

Of course, there's absolutely nothing to stop you writing the relevant Ruby code to talk SPI and blat a $global_variable but we unfortunately can't offer any official support for this for our users and also can't promise that this will continue to work from version to version. We can, however, officially support our OSC interface.

[icubex]

Hi @samaaron, Thanks very much for the detailed followup.

Using OSC involves more steps (and concepts) for a user than using some global variable or similar in Sonic Pi, that's all.

Triggering sounds with percussive gestures like tapping, such that the sounds are produced reliably with negligible latency after the tap, requires sensors to be sampled and processed at a higher rate than 20 Hz - at 20 Hz it's possible that a fast tap is not detected. For any useful processing of the sensor signal, much higher rates are required. For instance, the above Python script would not be able to measure impact force reliably at 50 Hz so as to calculate velocity - at least a couple of hundred Hz is needed. So it seems it's not feasible to do any significant sensor signal processing in Sonic Pi code. But even if the sensor signals are processed in eg. a Python script (instead of Sonic Pi code), the audio latency is too high - at least when running Sonic Pi on Raspberry Pi hardware. For us, the best option seems to continue working in Python with Pyo and it's very low latency.