Help Creating a Loopback Device

[leafac]

Hi @gavv (and other authors),

First, congratulations on the fantastic job! This library is awesome! I’m getting my feet wet in C++ and lower-level programming (I’m mostly used to web development, programming-language theory, and so forth), and I was getting a bit frustrated with the amount of complexity in https://developer.apple.com/documentation/coreaudio/creating_an_audio_server_driver_plug-in, but then I found libASPL and managed to get close to a working prototype in a couple hours 👏

I’d love if you could give me a couple pointers to continue.

I’m building a loopback device. Similar to https://github.com/ExistentialAudio/BlackHole, https://github.com/mattingalls/Soundflower, https://rogueamoeba.com/loopback/, and so forth. Here’s as far as I’ve managed to go:

Code

```cpp #include namespace { class Loopback : public aspl::IORequestHandler { public: void OnWriteMixedOutput(const std::shared_ptr& stream, Float64 zeroTimestamp, Float64 timestamp, const void* bytes, UInt32 bytesCount) override { for (auto bytesIndex = 0u; bytesIndex < bytesCount; bytesIndex++) { circularBuffer[circularBufferWriteIndex] = reinterpret_cast(bytes)[bytesIndex]; circularBufferWriteIndex++; if (circularBufferWriteIndex == circularBufferSize) circularBufferWriteIndex = 0; } } void OnReadClientInput(const std::shared_ptr& client, const std::shared_ptr& stream, Float64 zeroTimestamp, Float64 timestamp, void* bytes, UInt32 bytesCount) override { for (auto bytesIndex = 0u; bytesIndex < bytesCount; bytesIndex++) { reinterpret_cast(bytes)[bytesIndex] = circularBuffer[circularBufferReadIndex]; circularBufferReadIndex++; if (circularBufferReadIndex == circularBufferSize) circularBufferReadIndex = 0; } } private: UInt8 circularBuffer[48000 * 100]; UInt32 circularBufferSize = 48000 * 100; UInt32 circularBufferWriteIndex = 0; UInt32 circularBufferReadIndex = 0; }; } // namespace extern "C" void* EntryPoint(CFAllocatorRef allocator, CFUUIDRef typeUUID) { if (!CFEqual(typeUUID, kAudioServerPlugInTypeUUID)) return nullptr; auto context = std::make_shared(); aspl::DeviceParameters deviceParams; deviceParams.Name = "Loopback"; deviceParams.SampleRate = 48000; auto device = std::make_shared(context, deviceParams); device->AddStreamAsync(aspl::Direction::Input); device->AddStreamAsync(aspl::Direction::Output); device->SetIOHandler(std::make_shared()); auto plugin = std::make_shared(context); plugin->AddDevice(device); static auto driver = std::make_shared(context, plugin); return driver->GetReference(); } ```

To my surprise, audio is getting in and coming back out! 😁

But there are plenty of things I don’t understand yet:

1. Am I right in thinking that OnWriteMixedOutput() is called once per device, and that OnReadClientInput() is called once per client?

2. If so, should I have one circularBufferReadIndex per client?

3. Right now there’s a huge time gap between audio going in and coming back up. I suppose that’s because circularBufferWriteIndex and circularBufferReadIndex are out of alignment. So perhaps I shouldn’t have circularBufferReadIndexs at all? But then how would I keep track of where each client should be in the circularBuffer?

4. Doing all this circular buffer management by hand seems silly. For one thing, I suppose it’s far from being thread-safe. What data structure implementation do you recommend? Is this what libASPL’s DoubleBuffer is for?

5. My plan is to have a way for the user to create devices dynamically. I suppose it’s okay to create devices at any time and just AddDevice() and RemoveDevice() them as needed, right?

6. From what I read in libASPL’s README & CoreAudio/AudioServerPlugIn.h the plugin runs in a sandbox. What’s the best way to communicate with the plugin to add/remove devices? I was thinking of spinning up an HTTP server listening on a socket in the temporary directory right from the plugin process. Is this even viable? Is there a better idea?

7. Still related to the sandbox: How do I store the devices that should be created so that configuration is persisted across runs of Core Audio? Should I use the so-called “Storage Operations” in CoreAudio/AudioServerPlugIn.h? Is there an abstraction for it in libASPL that I couldn’t find?

8. How do I control bit depth? DeviceParameters has a way of controlling the sample rate, but not the bit depth…

9. Can libASPL synchronize the clock with other devices or will I run into issues similar to BlackHole?

Thank you very much in advance.

[gavv]

Hi Leandro!

Am I right in thinking that OnWriteMixedOutput() is called once per device, and that OnReadClientInput() is called once per client?

Yes. OnWriteMixedOutput is used if DeviceParameters::EnableMixing is true. If you set it to false, libASPL will instead use OnWriteClientOutput, and it will call it once per client. (Note: the mixing is done by CoreAudio, not by libASPL).

If so, should I have one circularBufferReadIndex per client?

Depends on your goals. If you disable mixing, you can store samples from each client in its own buffer.

Then you can for example do some routing, e.g. when client X reads samples, you may decide that it will read from buffer of client Y.

Or you can enable mixing and let all clients write to one shared buffer and read from the same buffer.

Right now there’s a huge time gap between audio going in and coming back up. I suppose that’s because circularBufferWriteIndex and circularBufferReadIndex are out of alignment. So perhaps I shouldn’t have circularBufferReadIndexs at all? But then how would I keep track of where each client should be in the circularBuffer?

Like 100 seconds (48000 * 100)? The size of your circular buffer is your latency. I don't think you can avoid using it at all, but you can make the buffer smaller.

Doing all this circular buffer management by hand seems silly. For one thing, I suppose it’s far from being thread-safe. What data structure implementation do you recommend? Is this what libASPL’s DoubleBuffer is for?

• TBH I don't remember if CoreAudio calls read and write operations on one thread or on different. I would assume that they all are called on same thread, and thus they don't need to be thread-safe. But don't take my word for it, better checks in docs or test it (you can print thread id from read and write callbacks and check if they're always the same). If you'll firgure this out, would be nice if you could share results.

• DoubleBuffer is a different thing. It's not a ring buffer, it stores a single value and provides lock-free getter, to avoid blocking real-time thread which calls that getter.

• Yes, there are numerous ready to use ring buffer implementations, quick googling gives some examples:

  • https://github.com/rmind/ringbuf
  • https://github.com/QuantumLeaps/lock-free-ring-buffer
  • https://github.com/jnk0le/Ring-Buffer
  • https://github.com/AndersKaloer/Ring-Buffer
  • https://github.com/dhess/c-ringbuf

  They provide different guarantees, some of them are not thread-safe, some are thread-safe, but not lock-free, some are lock-free but are SPSC, etc.

My plan is to have a way for the user to create devices dynamically. I suppose it’s okay to create devices at any time and just AddDevice() and RemoveDevice() them as needed, right?

Yep.

From what I read in libASPL’s README & CoreAudio/AudioServerPlugIn.h the plugin runs in a sandbox. What’s the best way to communicate with the plugin to add/remove devices? I was thinking of spinning up an HTTP server listening on a socket in the temporary directory right from the plugin process. Is this even viable? Is there a better idea?

This makes sense. You can also use gRPC or XPC (apple-specific thing). Personally I'd prefer gRPC in this specific case.

In general, all mechanisms based on sockets or shared memory should work.

Still related to the sandbox: How do I store the devices that should be created so that configuration is persisted across runs of Core Audio? Should I use the so-called “Storage Operations” in CoreAudio/AudioServerPlugIn.h? Is there an abstraction for it in libASPL that I couldn’t find?

• I don't know about this API, but from the comment it seems that's what you need: Note that the host provides a means for the plug-in to store and retrieve data from persistent storage.

• libASPL does not provide wrappers for this.

• It would be a suitable feature for libASPL though, so I'd be happy to accept a pull request :)

How do I control bit depth? DeviceParameters has a way of controlling the sample rate, but not the bit depth…

It's property of Format, which is property of Stream: https://github.com/gavv/libASPL/blob/main/include/aspl/Stream.hpp#L50

You may pass StreamParameters to Stream constructor, or use SetPhysicalFormatAsync() or SetAvailablePhysicalFormatsAsync().

Can libASPL synchronize the clock with other devices or will I run into issues similar to https://github.com/ExistentialAudio/BlackHole/discussions/27?

Nope, currently libASPL does not add any logic on top of CoreAudio and does not touch your samples.