search

PortAudio / portaudio

PortAudio is a cross-platform, open-source C language library for real-time audio input and output.
Other
1.39k stars 291 forks source link

Underflows when opening the same device twice #680

Open Martmists-GH opened 2 years ago

Martmists-GH commented 2 years ago

Describe the bug When opening the same device (in my case default/pulse) twice, whether two separate read/write streams or two separate read streams, underruns start happening at an alarming rate, as shown by ALSA lib pcm.c:8559:(snd_pcm_recover) underrun occurred in stderr.

My use case here is that pulseaudio has multiple sources/sinks, e.g. desktop audio and mic input, or a null sink and desktop audio, and these need to be handled as separate outputs so I can configure them with PAGraphControl, though maybe if portaudio properly supports sources/sinks as separate devices that could potentially solve this issue.

To Reproduce

  1. Open two streams to the same device in blocking mode
  2. Handle both as if nothing's out of the ordinary

Expected behavior Both streams act separately as they should deal with separate sinks

Actual behavior Underruns start happening and audio gets choppy and delayed

Desktop (please complete the following information):

Additional context The code for this can be found in the issue originally created at https://github.com/spatialaudio/python-sounddevice/issues/384

Martmists-GH commented 2 years ago

It seems the hostapi used is actually alsa with device pulse, sorry for providing the wrong information

Chum4k3r commented 2 years ago

Using ALSA's Pulse router adds several layers of pre processing of the audio data on each callback, plus you are using Python, which has its own performance limitations. I don't see any posible scenario of this settings work flawlessly.

You should try grab the device you wanna use from ALSA directly. The "pulse" device from ALSA hostapi Is not the same as using some device from PulseAudio hostapi. PulseAudio do support several sinks and sources, but ALSA don't

Em seg, 13 de dez de 2021 07:12, Martmists @.***> escreveu:

It seems the hostapi used is actually alsa with device pulse, sorry for providing the wrong information

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/PortAudio/portaudio/issues/680#issuecomment-992306597, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJPYUKF4C7DLLDSFUIOYO5DUQXBH3ANCNFSM5J5WRH2A .

Martmists-GH commented 2 years ago

Using ALSA's Pulse router adds several layers of pre processing of the audio data on each callback, plus you are using Python, which has its own performance limitations. I don't see any posible scenario of this settings work flawlessly.

It works fine when using a file and a single output, so the performance is not the issue here.

You should try grab the device you wanna use from ALSA directly.

I would love to, but pulseaudio also has virtual devices, and alsa doesn't show my mic/speakers/headphones at all, I only get pulse, pipewire (which shouldn't even exist) and default.

The "pulse" device from ALSA hostapi Is not the same as using some device from PulseAudio hostapi. PulseAudio do support several sinks and sources, but ALSA don't

How would I access pulseaudio as hostapi? It doesn't seem to exist in src/hostapis except on this fork from 10 years ago: https://github.com/bkgood/portaudio-pulseaudio

Chum4k3r commented 2 years ago

alsa doesn't show my mic/speakers/headphones at all

Usually, built-in hardware is shown at the top of device list, as a duplex device, e.g. it has input and output channels greater than 0. Can you show the output of python -m sounddevice? Be sure to not have any audio coming in or out from your computer, as this could mask the actual available channels on alsa side

Martmists-GH commented 2 years ago 
   0 HDA Intel PCH: HDMI 0 (hw:0,3), ALSA (0 in, 8 out)
   1 HDA Intel PCH: HDMI 1 (hw:0,7), ALSA (0 in, 8 out)
   2 HDA Intel PCH: HDMI 2 (hw:0,8), ALSA (0 in, 8 out)
   3 HDA Intel PCH: HDMI 3 (hw:0,9), ALSA (0 in, 8 out)
   4 HDA Intel PCH: HDMI 4 (hw:0,10), ALSA (0 in, 8 out)
   5 HDA NVidia: HDMI 0 (hw:1,3), ALSA (0 in, 8 out)
   6 HDA NVidia: HDMI 1 (hw:1,7), ALSA (0 in, 8 out)
   7 HDA NVidia: HDMI 2 (hw:1,8), ALSA (0 in, 8 out)
   8 HDA NVidia: HDMI 3 (hw:1,9), ALSA (0 in, 8 out)
   9 HDA NVidia: HDMI 4 (hw:1,10), ALSA (0 in, 8 out)
  10 HDA NVidia: HDMI 5 (hw:1,11), ALSA (0 in, 8 out)
  11 HDA NVidia: HDMI 6 (hw:1,12), ALSA (0 in, 8 out)
  12 hdmi, ALSA (0 in, 8 out)
  13 pipewire, ALSA (64 in, 64 out)
  14 pulse, ALSA (32 in, 32 out)
* 15 default, ALSA (32 in, 32 out)

It just shows a bunch of these random names, but I can't find "Built-in Audio" (microphone), "Built-in Audio Analog Stereo" (speaker) or "Null Output" (virtual sink/source)

If this is supposed to be alsa, it's also missing quite a lot of devices that are present in aplay -L and arecord -L

$ aplay -L
null
    Discard all samples (playback) or generate zero samples (capture)
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler
jack
    JACK Audio Connection Kit
oss
    Open Sound System
pipewire
    PipeWire Sound Server
pulse
    PulseAudio Sound Server
speex
    Plugin using Speex DSP (resample, agc, denoise, echo, dereverb)
upmix
    Plugin for channel upmix (4,6,8)
vdownmix
    Plugin for channel downmix (stereo) with a simple spacialization
default
    Default ALSA Output (currently PulseAudio Sound Server)
sysdefault:CARD=PCH
    HDA Intel PCH, CX11970 Analog
    Default Audio Device
front:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    Front output / input
surround21:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    4.0 Surround output to Front and Rear speakers
surround41:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
hdmi:CARD=PCH,DEV=0
    HDA Intel PCH, HDMI 0
    HDMI Audio Output
hdmi:CARD=PCH,DEV=1
    HDA Intel PCH, HDMI 1
    HDMI Audio Output
hdmi:CARD=PCH,DEV=2
    HDA Intel PCH, HDMI 2
    HDMI Audio Output
hdmi:CARD=PCH,DEV=3
    HDA Intel PCH, HDMI 3
    HDMI Audio Output
hdmi:CARD=PCH,DEV=4
    HDA Intel PCH, HDMI 4
    HDMI Audio Output
usbstream:CARD=PCH
    HDA Intel PCH
    USB Stream Output
hdmi:CARD=NVidia,DEV=0
    HDA NVidia, HDMI 0
    HDMI Audio Output
hdmi:CARD=NVidia,DEV=1
    HDA NVidia, HDMI 1
    HDMI Audio Output
hdmi:CARD=NVidia,DEV=2
    HDA NVidia, HDMI 2
    HDMI Audio Output
hdmi:CARD=NVidia,DEV=3
    HDA NVidia, HDMI 3
    HDMI Audio Output
hdmi:CARD=NVidia,DEV=4
    HDA NVidia, HDMI 4
    HDMI Audio Output
hdmi:CARD=NVidia,DEV=5
    HDA NVidia, HDMI 5
    HDMI Audio Output
hdmi:CARD=NVidia,DEV=6
    HDA NVidia, HDMI 6
    HDMI Audio Output
usbstream:CARD=NVidia
    HDA NVidia
    USB Stream Output
$ arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler
jack
    JACK Audio Connection Kit
oss
    Open Sound System
pipewire
    PipeWire Sound Server
pulse
    PulseAudio Sound Server
speex
    Plugin using Speex DSP (resample, agc, denoise, echo, dereverb)
upmix
    Plugin for channel upmix (4,6,8)
vdownmix
    Plugin for channel downmix (stereo) with a simple spacialization
default
    Default ALSA Output (currently PulseAudio Sound Server)
sysdefault:CARD=PCH
    HDA Intel PCH, CX11970 Analog
    Default Audio Device
front:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    Front output / input
usbstream:CARD=PCH
    HDA Intel PCH
    USB Stream Output
usbstream:CARD=NVidia
    HDA NVidia
    USB Stream Output
Chum4k3r commented 2 years ago

From the looks, this one should be presented as your built-in audio hardware by alsa.

front:CARD=PCH,DEV=0
    HDA Intel PCH, CX11970 Analog
    Front output / input

Can you build the example code at https://github.com/PortAudio/portaudio/blob/master/examples/pa_devs.c to see if the hardware is present in its output? This would help narrow if it is a PortAudio problem or a Python/SoundDevice problem.

Martmists-GH commented 2 years ago

Output of pa_devs: log.txt It doesn't seem to find those outputs either.

Also, is pulseaudio support not planned at all or is there an ETA?

Chum4k3r commented 2 years ago

There is a PR #336 that is intended on adding support for pulseaudio, but I don't know how close to done it is

Chum4k3r commented 2 years ago

Output of pa_devs: log.txt

PortAudio seems to be unable to enumerate the audio hardware

Martmists-GH commented 2 years ago

How would I solve that?

RossBencina commented 2 years ago

@martmists-gh wrote::

  1. Open two streams to the same device in blocking mode
  2. Handle both as if nothing's out of the ordinary

I'm not sure what "Handle both as if nothing's out of the ordinary" means, but I think you're going to have problems if you block on one stream while the other is starved for data. You should make sure to call Pa_GetStreamReadAvailable or Pa_GetStreamWriteAvailable and only access that many samples to avoid blocking.

Martmists-GH commented 2 years ago

I'm already doing that so there shouldn't be any issues there.

philburk commented 2 years ago

I suggest starting with a phase where you try to fill both streams. Write as many zeros as you can to both streams based on Pa_GetStreamWriteAvailable(). When they are both running full and stable then start writing real data.

RossBencina commented 2 years ago

You posted the code here: https://github.com/spatialaudio/python-sounddevice/issues/384 I don't see any call to Pa_GetStreamWriteAvailable or equivalent.

I don't think we can offer support for this without a reproducible test case in C. Even then I'm not sure who would be able to fix it, but if you can provide a test case we could continue the discussion.

Alternatively, if you would like to post a complete, minimal test case in Python maybe someone can suggest something.

Martmists-GH commented 2 years ago

The current code can be found at https://github.com/martmists-gh/kaudio-python which should give the same issues. Simply place an input and output node and connect them.

philburk commented 2 years ago

Have you tried portaudio/test/patest_wire.c on your device? It does full duplex.

RossBencina commented 2 years ago

It's unlikely that anyone is going to debug your application code for you. I recommend posting a minimal/reduced single-file test case that uses the minimal amount of code to reproduce the problem, preferably in C like our existing tests.

It's unclear whether this is actually a bug in PortAudio. Synchronising multiple blocking streams is non-trivial, but definitely involves using the Pa_GetStream{Read/Write}Available APIs to ensure that buffer fill levels are maintained, since there is no inherent synchronization between PA streams.

Martmists-GH commented 2 years ago

Have you tried portaudio/test/patest_wire.c on your device? It does full duplex.

That one works fine, the difference between that one and mine is that I open the two streams separately as part of my application rather than opening both input and output on the same stream.