Mix several sounds in a Stream

[cyberic99]

Hello

Is it possible to keep a Stream running, and play simultaneous sounds in it?

Otherwise, is there a way to play a sound with a specific volume?

Thank you

[irmen]

I don't think this library supports that directly however you could do the mixing yourself and then write the mixed sound data to the player stream.

This is what I do in my synthplayer library (which just got support for the soundcard library as backend) It allows you to do a lot of stuff such as playing samples at the same time (it mixes them for you):

import time
from synthplayer.sample import Sample
from synthplayer.playback import Output

s1 = Sample("samples/909_clap.wav").normalize()
s2 = Sample("samples/909_mid_tom.wav").normalize()
s3 = Sample("samples/909_crash.wav").normalize()

with Output() as out:
    out.play_sample(s1)
    time.sleep(0.1)
    out.play_sample(s2)
    time.sleep(0.1)
    out.play_sample(s3)
    time.sleep(0.1)
    out.wait_all_played()

[bastibe]

This is mostly right. SoundCard does not mix streams, and each player can only play one stream at a time. However, you can open multiple players in multiple threads or processes, which will play simultaneously.

Additionally, the high-level (non-streaming) play and record functions return immediately, which allows for (limited) simultaneous playback of multiple sounds.

[irmen]

@bastibe interesting, good to know. Never tried to open multiple streams concurrently yet.

[cyberic99]

@irmen thank you, I tried your lib and the samples are playing together correctly.

However, I try to play some samples synced to an external MIDI device, and I can notice a big latency, meaning that the audio sounds are late compared to the external MIDI sounds... even if I preload the samples...

I tried several libs: pygame, pyglet... I couldn't find any lib with a tiny enough latency...

I'll try to use soundcard with threads

[cyberic99]

@bastibe I tried to play multiple sounds, each one in a thread.

After 2 or 3 sounds played correctly, I get an error:

Assertion 's' failed at pulse/stream.c:1392, function pa_stream_connect_playback(). Aborting.

I just call speaker.play(data, 48000), data being a numpy array

The sounds play at a relatively fast rate, ie there are several mixed sounds per second

[bastibe]

I have seen this issue before, but haven't been able to figure out its reason. But your comment prompted me to revisit the issue, and I think I have found a solution.

Could you check if the latest commit fixes your issue?

[cyberic99]

Hi @bastibe !

With your latest commit, it is way better.

But after a few (3 or 4) runs, I encountered the problem again.

For each run I play 90 samples in 12 seconds

Is there a way to get more debug ? (apart compiling pulseaudi in debug mode)...

[cyberic99]

Also, sometimes, I get this error:

    with speaker.player(44100, channels=2, blocksize=(int)(512/16)) as spp:
  File "soundcard/pulseaudio.py", line 571, in __enter__
    raise RuntimeError('invalid channel map', str(channelmap))
RuntimeError: ('invalid channel map', "<cdata 'pa_channel_map *' 0x7fd74c0019a0>")

(I modified the code to print the channel map)

It happens when playing on of the first 10 sounds or so. but the other sounds seem to play anyway

Do I have to create a new player instance in each thread?

[bastibe]

But after a few (3 or 4) runs, I encountered the problem again.

For each run I play 90 samples in 12 seconds

Is there a way to get more debug ?

I think this issue is caused by opening too many pulseaudio contexts or runloops. Originally, I opened one run loop per player, as the pulseaudio docs, ever helpful, did not give any indication that this would be a problem. As it turns out, I did find one little remark after all that mentions that you shouldn't use more than one context per application. So in the latest commit, the run loop and context is global, and is re-used by every player.

If the issue still persists, I would expect that it is still opening multiple run loops or contexts somehow. Are you importing SoundCard in multiple independent threads or processes? Or are you using anything else that might reset the SoundCard module (like the autoreload extension for IPython)?

Another possibility would be that I forgot to to lock the main loop somewhere in the code. SoundCard is using the threaded sync main loop, which needs to be locked before every interaction. Do you see any kind of regularity in the source of the errors?

As for your channel map issue, I honestly have no idea. The code just instantiates a default channel map without modifying it at all at that point in the code. I suspect this to be a subtle bug in pulseaudio, but I am not sure. Maybe it's a locking issue, as described above?

[cyberic99]

If the issue still persists, I would expect that it is still opening multiple run loops or contexts somehow. Are you importing SoundCard in multiple independent threads or processes?

No. I import soundcard, then create a new thread which will be responsible for playing only its own sound

Or are you using anything else that might reset the SoundCard module (like the autoreload extension for IPython)?

I don't think so.

I will have a closer look at my code and try to reproduce the problem with a simple example.

Do you think I should add a print somewhere in SoundCard' s init?

[bastibe]

Do you think I should add a print somewhere in SoundCard' s init?

That's a great idea! You could add a print in _PulseAudio.__init__ and see if it gets called more than once. It should only be called once.

[cyberic99]

Unfortunately the init is only called once...

Maybe it is simple a bug in pulseaudio?

The version 5.0 that I am using is quite old....

[bastibe]

Could you create a short example script that triggers the issue on your system? Then we could see if this is a bug on your system, or if it is reproducible with different hardware and different versions of pulseaudio.

[irmen]

@cyberic99 if you're still using my synthplayer library with the soundcard as a backend api, try switching to another api supported by synthplayer to see if the issue really is in soundcard

[cyberic99]

@irmen in fact I tried several libraries, my goal was to achieve the lowest possible latency

I did try several backends, some of them would not work, some of them produced no sound...

But I didn't experience the assertion problem.

I'll try to code a simple repro case soon.

[irmen]

While I personally believe Python should be able to provide microsecond level latency, it may be wise to explore other options (writing more of the audio processing logic in C?).

[cyberic99]

@irmen I too believe ot is possible. Python can be surprisingly fast sometimes!

@bastibe I couldn't reproduce this exact iasue on another computer with a more modern distro.

so I think this issue can be closed for now. I will open another one, that is actually a feature request.

Thank you all for your help!

[bastibe]

@irmen All audio playback/recording necessarily has to use the OS's audio libraries underneath, which is usually written in C anyway. Where Python comes in is with reading/writing data to the C API. SoundCard reads/writes this data as NumPy arrays, which maximizes usability on the Python side, instead of raw speed.

If all you do is play the data received from record, SoundCard can use block lengths down to four samples on a modern computer (based on a test I did many months ago). I think this is entirely reasonable. However, this will quickly fall apart once you try to process the data in any way, but then that is out of SoundCard's hands.

In Python, good latency is at odds with good performance, since one requires short arrays, while the other requires long arrays. Nevertheless, block lengths in the low hundreds of samples, or low tens of milliseconds, work well, usually.