Record pecks during playback

[timsainb]

I would like to record pecks during playback of audio- and allow the audio to be killed early if the bird makes an early decision.

[neuromusic]

Ostensibly pyoperant supports this, but implementation is going to be highly dependent on hardware.

Currently, playback using the AudioOutput requires three methods.

• audio.queue('filename.wav') reads the file and loads it into the pyaudio buffer
• audio.play() plays the audio without blocking
• audio.stop() stops the audio

after audio.play(), the TwoAltChoice protocol waits a minimum wait time https://github.com/gentnerlab/pyoperant/blob/master/pyoperant/behavior/two_alt_choice.py#L427 which by default is the length of the stimulus

The short answer is to adjust the min_wait variable in trial_pre() https://github.com/gentnerlab/pyoperant/blob/a46e96e4f55989a8927733ef44ccd1b94d7d45e0/pyoperant/behavior/two_alt_choice.py#L327 and add whatever logic you need to keep track of the wav file's early termination.

Practically, however, you'll run into some problems.

Audio playback is fairly CPU-heavy. Polling for pecks with comedi is VERY CPU heavy. In my experience, allowing early responses ended up causing the audio to drop while pyoperant was polling for responses. This is largely a problem with the limitations of the comedi drivers we are using and the polling mechanism.

Better approaches

take advantage of PyAudio's callback feature

PyAudio supports PortAudio's callback which executes on each chunk of audio. This needs to be a short, fast function, however. See limitations here: https://www.assembla.com/spaces/portaudio/wiki/Tips_Callbacks

use PyDAQmx instead of comedi

NI hardware which supports NIDAQmx can be used to execute polling more efficiently, as the polling takes place on the hardware and sends a callback to the software when there is a state change on a port(s).

The "right way" to do this seems to be:

1. use NIDAQmx to poll for responses and drop them into an event queue.
2. on each PyAudio callback, check the queue for recent responses

[siriuslee]

We have been doing exactly what @neuromusic suggested with no real issues using PyAudio for playback and an Arduino for polling. I've nearly finished my implementation of a nidaq interface using pylibnidaqmx and I'll try to test the same use case when I do.

[Jeffknowles]

Hey All!

Good work on making pyoperant an easy to use tool!

One suggestion re standard recording of events regardless of state machine is to separate out the event 'catcher' from the state machine. Hardware should run as a separate loop (or loops depending on the devices involved) that feed events to a main event queue in python. Lower level loops on the hardware layer can be written to debounce, edge detect ext.

Something like this is implemented in my behavior controller code for several different types of hardware (event detector running on arduino, event detector using c-python libraries for beaglebone, pi GPIO, ext; comedi and nidaqmx). If its helpful you can check out how I have it implemented and steal as useful!

https://bitbucket.org/spikeCoder/kranky/src

Best,

Jeff

On Tue, Mar 29, 2016 at 9:09 AM, siriuslee notifications@github.com wrote:

We have been doing exactly what @neuromusic https://github.com/neuromusic suggested with no real issues using PyAudio for playback and an Arduino for polling. I've nearly finished my implementation of a nidaq interface using pylibnidaqmx and I'll try to test the same use case when I do.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/gentnerlab/pyoperant/issues/110#issuecomment-202977734

Jeff Knowles

[neuromusic]

sweet. thanks @Jeffknowles this might be nice for some of the refactoring

@siriuslee - are you using the 'callback' attribute I added to the pyaudio interface? or did you write your own portaudio/pyaudio callback? I'm considering getting rid of my callback "feature" in favor of letting users define the function that pyaudio wants. it seems less risky to force people to at least read the portaudio documentation so they know how it works (and the caveats/risks)

see https://github.com/gentnerlab/pyoperant/pull/111 for my proposed changes

[siriuslee]

We didn't require any use of the callback function. Since pyaudio plays sound without blocking, we just kick off a poll of the pecking port for the duration of the sound. Then, if the poll exits before timing out, we call speaker.stop().