Sounds sometimes play at wrong sample rate

[cxrodgers]

Every once in a while, one of my Pis will just start playing sound at the wrong sample rate. I would guess it is playing at 48Khz instead of 192Khz, because the stimuli are several times slower and several times lower-pitched than they should be. Restarting autopilot doesn't fix it, but restarting the pi usually does.

I'm confused why this is happening, I have checked that jackd is started using the normal parameters, and that the autopilot software believes the sample rate is 192KHz everywhere this variable is used. It's as if the downstream software (alsamixer?) is stuck on the wrong sample rate. Is there any way to check what the hardware thinks its current sample rate is?

[sneakers-the-rat]

This could happen if jack audio is already running, for example if it was installed by apt or if some other service starts it automatically.

The sound server is started by a remarkably unsophisticated script here: https://github.com/auto-pi-lot/autopilot/blob/04b5968ba02c8a1413a27eb6a138b6a186b130f1/autopilot/external/__init__.py#L92

so you should get a warning if it's already running, and the soundserver should detect what sample rate the jack server is at https://github.com/auto-pi-lot/autopilot/blob/04b5968ba02c8a1413a27eb6a138b6a186b130f1/autopilot/stim/sound/jackclient.py#L171 and should raise a warning if it doesn't match what is set in prefs: https://github.com/auto-pi-lot/autopilot/blob/04b5968ba02c8a1413a27eb6a138b6a186b130f1/autopilot/stim/sound/jackclient.py#L208-L212

Then the sound classes will be default get the sampling rate set by the jackclient module (which is super flimsy and i want to bring into the realms of the new prefs model which is going to be file-based rather than runtime based) https://github.com/auto-pi-lot/autopilot/blob/04b5968ba02c8a1413a27eb6a138b6a186b130f1/autopilot/stim/sound/base.py#L217

which is copied from the server when it boots: https://github.com/auto-pi-lot/autopilot/blob/04b5968ba02c8a1413a27eb6a138b6a186b130f1/autopilot/stim/sound/jackclient.py#L196

and that's used throughout the synthesizing process.

So it should propagate up through the jackd process, I would check the stderr logs during the process startup from the autopilot daemon if you are using it that way. Using the jack client in python is a pretty straightforward way of directly querying the sound server specifically about sampling rate.

The other thing might be if there was some kind of resource limitation and the server was getting starved for cpu cycles, but you should see buffer underruns in the stderr if that is the case.

See also: https://docs.auto-pi-lot.com/en/latest/stim/sound/jackclient.html#autopilot.stim.sound.jackclient.JackClient

[cxrodgers]

Thanks! I have traced through the locations you mentioned and Fs is always showing up properly as 192K, yet it stubbornly plays more slowly. I think it must be an alsa or jack problem, not an autopilot issue.

In the interim, I wrote a short thing that will detect how fast JackClient is actually able to write frames to the buffer, which is a good indicator of when the problem is happening. So at least I can raise an error when it does.

[cxrodgers]

Figure this out!! The problem was that we were occasionally running this line in between Autopilot runs, in order to do something unrelated to Autopilot: sudo pigpiod -x 1111110000111111111111110000

We should have been running this: sudo pigpiod -t 0 -l -x 1111110000111111111111110000

Without -t 0, pigpiod captures the PWM clock, leaving it unavailable for other services like audio. https://abyz.me.uk/rpi/pigpio/pigpiod.html I don't know if the missing -l affected anything.

What is strange is that this problem persists until the next reboot, even though Autopilot always does a sudo killall pigpiod and then uses a proper invocation of pigpiod. Possibly killing pigpiod in this way doesn't tell it to let go of the PWM clock.

[sneakers-the-rat]

ah yes - frick I should have known, it's that by default: https://github.com/auto-pi-lot/autopilot/blob/04b5968ba02c8a1413a27eb6a138b6a186b130f1/autopilot/prefs.py#L351 but i have yet to refactor the prefs so that they are individually documented/documentable. I took note of this on the pigpio page: https://wiki.auto-pi-lot.com/index.php/Pigpio

pigpio I think has outlived its usefulness, and I want to rework the GPIO classes so you can use different backends - managing the pigpio daemon has proven to be a huge pain in the ass, and for most cases that don't use more demanding features like PWM and waveforms or scripts and whatever it is very much not worth it. I almost always have to check with top or htop that is actually dead and typically resort to rebooting. I haven't found a clean and reliable way to manage the process or end it, despite having tried about a zillion types of trying to make sure all our references and use of its resources are cleaned up.

The short term fix for this is fixing prefs so that they aren't just untyped dicts, the medium term fix for this is to clean up the API of the GPIO classes so they are more predictable and then implement different backends as mixins that depend on different prefs (eg. the prefs should be declarative and actually be able to check that they are truly set, so when one sets an option in pigpio, if you have run it in a different process and it is not using the argument you pass to PIGPIODARGS then you can know that, and if you specify one backend vs. another it is possible to know how to satisfy the environmental dependencies for that). the longterm fix is to break this project up into interoperable component projects so that it is easier to chase down each of these individual pieces rather than them getting lost in a big haystack.

reopen this if it comes back <3