Not an issue just a call for a little help

[StuartIanNaylor]

Really like sounddevice as use the callback with tensorflow audio classification quite a lot, but I am stuck on how to implement the callback methods with say a asyncio co-routine such as websockets.

I have been googling and I can not find any good examples of melding both together, can find a plethora of examples for both standalone.

I have a very basic websockets server @ https://github.com/StuartIanNaylor/ProjectEars/blob/main/dataset/kws/server.py and I am totally stuck at the very basics as can not figure out how the callback and asyncio should be employed. I am in a state of confused.com and thought anyone got any exiting source code and examples of the 2 that I can use as a basis as I am really stuck where to start with this one.

Apols about in issues but hoping someone might be able to nudge me in the right direction as would I be better dropping the callback and implementing a blocking call but with asyncio?

[HaHeho]

Performing audio classification means you will only need sound input / capture (for now)?

I would say building something where sounddevice captures into a buffering thread-safe FIFO queue (e.g. in https://github.com/spatialaudio/python-sounddevice/blob/master/examples/rec_unlimited.py) and your async functions take out the should be possible. Although the data will be sent over network / internet to a different machine via websockets? Either way, that should be possible with some buffers in-between. Then it is a matter of tuning the respective buffer sizes and queue lengths so the application runs smoothly.

Hope that helps as a first idea. There may be better ones. Other people that have actual experience with async and/or tensorflow are probably also around.

[StuartIanNaylor]

Thnx for the reply I will give it another go but having a total brain freeze with asyncio and a callback, likely the websocket asycio should just add a queue item and the callback pull the item from the queue. I think I will just queue up a chunk of np.zeros so the callback remains 1 chunk behind the receive queue : anyway... I was hoping someone would say this app does that and have a look at, as not much with both is turning up in a google. Thnx though

[HaHeho]

There is the two asyncio examples actually. Probably a better starting point, but not sure which one suits you better. https://github.com/spatialaudio/python-sounddevice/blob/master/examples/asyncio_coroutines.py https://github.com/spatialaudio/python-sounddevice/blob/master/examples/asyncio_generators.py

[StuartIanNaylor]

Yeah https://github.com/spatialaudio/python-sounddevice/blob/master/examples/asyncio_coroutines.py confused me as unless I have this wrong its an asyncio example that runs sequentially. It actually records in full and then plays back in full and would seem a strange choice of example for asyncio.

That example is part of my confusion.

[mgeier]

Yeah https://github.com/spatialaudio/python-sounddevice/blob/master/examples/asyncio_coroutines.py confused me as unless I have this wrong its an asyncio example that runs sequentially.

Yes, that's true.

It actually records in full and then plays back in full and would seem a strange choice of example for asyncio.

Well, it's just a contrived example.

The module docstring says: "This example shows how to create a stream in a coroutine and how to wait for the completion of the stream."

And it does that twice in a row.

For me, the interesting part of this example is how to wait for the completion.

But you can of course also do something else concurrently while waiting for the completion. I didn't want to complicate the example, but if you have ideas how to improve it, please let me know!

[StuartIanNaylor]

Unfortunately no as was rather hoping for an example as I was looking at recording and also simultaneously sending via websockets. How to meld a callback with asyncio is where I get confused and if really its better to use threading. An async example would be great as a synchronous example using asyncio doesn't seem to make much sense :)

The examples are usually templates I can use but this one prob isn't worth using and that was as feedback nothing more.

[mgeier]

How to meld a callback with asyncio is where I get confused

You can have a look at the inputstream_generator() (in the asyncio_generators.py example). If you don't want to use a generator, you can just replace the yield line with (the hypothetical) await send_to_websocket(audio_data).

This wouldn't handle backpressure (i.e. the queue would grow indefinitely if the client doesn't read fast enough), but it should be enough for a first prototype.

The important point is that you use a queue to get the data from the audio callback to the async function where you send the data over a websocket. Same on the receiving end: read the data from the websocket into a queue and read from that queue in the audio callback to send it to the sound card.