audio glitches when I2C or SPI display is updated

todbot commented 1 year ago

CircuitPython version

Adafruit CircuitPython 8.0.0-beta.4-78-g4418f268b on 2022-12-06; Raspberry Pi Pico with rp2040

Code/REPL

# audio_click_i2c.py -- demonstrate audio clicking when update I2C display
import time, random
import board, busio
import audiocore, audiomixer, audiobusio, audiopwmio
import displayio, terminalio
from adafruit_display_text import label
import adafruit_displayio_ssd1306

displayio.release_displays()

dw,dh = 128,64
# Pico
disp_sda_pin = board.GP18
disp_scl_pin = board.GP19
audio_pin = board.GP0

disp_i2c = busio.I2C( scl=disp_scl_pin, sda=disp_sda_pin) # frequency=400_000 )
display_bus = displayio.I2CDisplay(disp_i2c, device_address=0x3C)  # or 0x3D depending on display
display = adafruit_displayio_ssd1306.SSD1306(display_bus, width=dw, height=dh)

maingroup = displayio.Group()
display.show(maingroup)
text_area = label.Label( terminalio.FONT, text="hello", color=0xFFFFFF, x=18, y=dh // 2 - 1, scale=2)
maingroup.append(text_area)

audio = audiopwmio.PWMAudioOut(audio_pin)
mixer = audiomixer.Mixer(voice_count=2, sample_rate=22050, channel_count=1,
                         bits_per_sample=16, samples_signed=True)
audio.play(mixer) # attach mixer to audio playback, starts audio going

print("audio engine now playing")

wave1 = audiocore.WaveFile(open("wav/snowpeaks_22k_s16.wav","rb"))
mixer.voice[1].play( wave1, loop=True )
mixer.voice[1].level = 0.5  # half volume

last_time = time.monotonic()
while True:
    print("hello")
    if time.monotonic() - last_time > 0.5:
        last_time = time.monotonic()
        text_area.x += 1
    time.sleep(0.1)

Behavior

Updating a I2C or SPI display on while audio (PWM or I2S) is playing causes that audio to loudly glitch.

Behavior demonstrated on three different setups in attached video:

https://user-images.githubusercontent.com/274093/206587929-599126b6-5484-49e2-87a7-78b52a03180b.mp4

Description

Glitching occurs for both I2C-based or SPI-displays, both OLED and TFT
Glitching occurs for both PWM- and I2C-based audio outputs
Lower I2C or SPI bus speeds cause a longer audible glitch
Playing audio direct (audio.play()) and via AudioMixer (mixer.voice[0].play()) causes glitch
If using AudioMixer, don't even need to play WAV file, glitching occurs when silent too

Tested on:

CircuitPython 8.0.0-beta.4-78-g4418f268b on Pico RP2040 w/ PWM audio
CircuitPython 8.0.0-beta.4-78-g4418f268b on Pico RP2040 w/ I2S audio
CircuitPython 7.3.3 on ItsyBitsy RP2040 w/ PWM audio

Issue #6439 might also be related to this

Additional information

No response

tannewt commented 1 year ago

Can you use larger audio buffers? This is likely that the display update takes longer than playing back an audio buffer.

todbot commented 1 year ago

Okay I feel like a total dummy for forgetting that audiomixer.Mixer() had buffer_size argument.

You are correct: setting buffer_size=32768 allows even the slowest 100kHz I2C bus full-screen OLED update to occur without glitching. At 400kHz I2C, only an 8kB buffer is needed. This is for for 22kHz mono 16-bit samples.

Adding the larger buffer also solves one of my most headache-inducing problems that still exists even after #6005 was closed: audio glitches caused by USB enumeration on reset.

I feel like I can close this issue, but is there somewhere official the tip of "if you get audio glitches, increase buffer size" can be documented?

dhalbert commented 1 year ago

I feel like I can close this issue, but is there somewhere official the tip of "if you get audio glitches, increase buffer size" can be documented?

There is no general audio Learn Guide, though there could be. This kind of info could also be in the FAQ in the Welcome guide.

Have you heard playback glitches when not using Mixer? I wonder if other default buffer sizes might be increased, at least on large-ram builds. Or we could provide optional parameters.

todbot commented 1 year ago

Have you heard playback glitches when not using Mixer? I wonder if other default buffer sizes might be increased, at least on large-ram builds. Or we could provide optional parameters.

Yes, if you update a I2C/SPI display without AudioMixer it glitches in the same way.

jepler commented 1 year ago

That can be somewhat ameliorated by doing explicit (not automatic) display refreshes. Refreshing the display from background blocks other background tasks for the whole time. I'm not sure but I thought that audio background task can take place during foreground display refresh. Ensuring refresh areas are small helps too of course

todbot commented 1 year ago

@jepler I was not able to find a way of doing explicit display refreshes that minimized the glitching at all. If you have an example I’d love to see it.

jepler commented 1 year ago

My information may not be accurate, it's more a memory than real facts.

jepler commented 1 year ago

In the very specific case that you're

drawing something that uses an ondiskbitmap
and it's calling back into python code to read the disk (adafruit_sdcard)
and you do a foreground refresh of the display, not a background refresh you can get background tasks running during the call out to adafruit_sdcard.

aalary commented 1 year ago

I am also seeing this problem with I2S audio and an I2C screen.

Adafruit CircuitPython 7.3.3 on 2022-08-29; Raspberry Pi Pico with rp2040 Board ID:raspberry_pi_pico

import adafruit_displayio_ssd1306

Increasing the buffer_size in audiomixer.Mixer() seems to eliminate the audio glitches, but it introduces a significant amount of latency that interferes with real time operation. Up to about 350ms of latency for buffer_size=32768 and sample_rate=44100 compared to about 15ms with buffer_size=1024.

agustinmista commented 6 months ago

Hey,

I'm experiencing the same issue as @aalary: a big tradeoff between glitches, sampling rate and latency.

My current setup is:

CircuitPython: Adafruit CircuitPython 8.2.9 on 2023-12-06; Raspberry Pi Pico with rp2040
Board: Raspberry Pi PICO
RP2040 frequency: 240_000_000
I2C frequency: 1_000_000
Synthio sampling rate: 44_100
Audiomixer buffer size: 4096

This seems to work fine so far, although I would like to reduce the buffer size a bit further to avoid latency. So, I was thinking: would it be possible to expose the state of the audio buffer somehow? This way we could perhaps delay display updates if the buffer is, say, less than 75% full.

todbot commented 6 months ago

@agustinmista, try lowering your audio sample rate and your MP3 bit rate. The RP2040 cannot really handle such high-quality audio, even when overclocked. I usually use 22050 Hz for WAV files. For MP3 files, you'll likely have to go lower (as mentioned in the audiomp3 docs) because it's decoding MP3s in the CPU instead of using hardware acceleration.

agustinmista commented 6 months ago

@todbot thanks for the heads up. I'm not playing MP3s but MIDI notes using synthio and, indeed, 22_040/2_048 seems to work without glitches if I keep the OLED UI very minimal.

Ah, and thanks a lot for the fantastic tricks repo! 😄

bgberk commented 3 months ago

Can confirm that this works with other SPI devices: I was unable to mix audio without crashing when loading WAVs from a microSD reader. Increased the buffer size and am now able to load and play as usual.

What determines the max size I could set the buffer to on, say, a Pico W?

adafruit / circuitpython