search

bastibe / SoundCard

A Pure-Python Real-Time Audio Library
https://soundcard.readthedocs.io
BSD 3-Clause "New" or "Revised" License
680 stars 69 forks source link

Provide asynchronous recording interface #67

Closed Eelis closed 4 years ago

Eelis commented 4 years ago

Recorder's record() method "will wait until numframes frames have been recorded". It would be great if the library provided an async Awaitable so that user code could choose to wait asynchronously for the recorded frames.

bastibe commented 4 years ago

SoundCard heavily depends on the operating systems' native audio APIs. In some APIs, SoundCard's functions will be called by the APIs on their own thread. Some APIs require SoundCard's functions to run at least once per block. If these requirements are not honored, you will get partial data, overflows, or crashes.

Async is nice if all of your functions are happy to wait arbitrary lengths of time for their time to run. Audio is not such a case. Audio is not asynchronous, but concurrent. Audio is inherently real-time. If you don't provide audio data on time, every time, then you won't have any audio to play and you'll hear popping and clicking. If you don't fetch your audio data in time, it will overflow and be lost forever.

Thus, SoundCard can not use async. If you need to do something else while SoundCard is doing it's thing, you can try doing it in a thread. But beware that you don't starve SoundCard's thread or you'll lose data. If that's a problem, run the audio processing in its own process.

That's just the reality of doing audio in Python. Sadly.

Eelis commented 4 years ago

I see, interesting, thanks for the explanation!

That's just the reality of doing audio in Python.

Well, if one only cared about Linux, couldn't one wrap PulseAudio's native async interface in Python using the Python async framework?

bastibe commented 4 years ago

Well, if one only cared about Linux, couldn't one wrap PulseAudio's native async interface in Python using the Python async framework?

Not at all. Pulseaudio is not "async" in the Python sense. It is not cooperative multitasking, where each concurrent process willingly relinquishes control at somewhat regular intervals. Instead, you have to register callbacks, which are called by Pulseaudio when it requires data or has data available.

Which is in fact what SoundCard is doing on Linux.

But, again, the problem is not any one API, it's audio in general. Audio is inherently real-time. You need to provide your data on time, every time. Thus, any cooperative scheme (i.e. async) is simply not viable. If pulse needs audio data, it needs your audio data now. It can not wait for another async function to give up control of its own volition.

As Python is, for the purpose of audio processing, single-threaded, this is only possible with preemptive multitasking, i.e. threads. With threads, the audio engine can interrupt every other thread when the data is available, and resume them when the data has been dealt with. It does not need to wait for the next await.

Eelis commented 4 years ago

Ahh, I see! Ok, that explains how the nature of Python plays into it. :) Thanks again!