Handling different audio data layouts (interleaved / non-interleaved)

[mitchmindtree]

Currently CPAL's stream data callback API allows the user to assume that audio data will be provided with channels interleaved in a single slice of memory. There are some issues with this:

• Not all hosts provide an option for emitting data in an interleaved layout. It's also unclear whether some hosts provide non-interleaved channels within contiguous memory. As a result, CPAL currently does the necessary conversion to and from interleaved for these hosts under the hood - a non-trivial cost for energy/peformance-sensitive applications.
• Different downstream applications prefer different layouts, and some hosts allow for specifying a desired layout. E.g. I believe CoreAudio provides an API for requesting interleaved/non-interleaved, though I don't believe there is any guarantee that either is supported, just that at least one of them will be.

We should update CPAL's API guided by the following broad goals:

• Support both interleaved and non-interleaved layouts, while keeping in mind the potential for other more obscure layouts in the future. For now at least we can be sure that each supported host provides at least interleaved and/or non-interleaved.
• Allow users to query if a layout is supported by a device.
• Allow for users to request a particular layout when building a stream.
• Allow for users to provide a callback or callbacks for handling data in different layouts depending on what is supported by the device.

I don't have a solid proposal in mind yet, though thought I'd open this so that we have a place to discuss.

[Ralith]

Straw proposal:

pub struct Format {
    pub channels: ChannelCount,
    pub sample_rate: SampleRate,
    pub data_type: SampleFormat,
    pub interleaved: bool,
}

impl Data {
    pub fn as_slice<T>(&self) -> Option<&[&[T]]> { ... }
}

[mitchmindtree]

I like this as a nice simple solution for Data and the build_input/output_stream_raw methods.

I wonder how we should handle this in the build_input/output_stream methods?

Perhaps rather than taking &[&[T]] and requiring the user must check the layout on every call, we take a similar approach to #119 and require specifying the layout as a type. I'm imagining |data: &Interleaved<T>| ... where Interleaved<T> derefs to &[T], and NonInterleaved<T> derefs to &[&[T]].

[Ralith]

I don't think that materially impacts the amount of checking the user has to do in the deinterleaved case. It's low-friction to access channel indices up to the number you requested, which are guaranteed to be there if the create call succeeded. It's definitely a little nicer for the interleaved case to not have to sprinkle [0] everywhere, though, and embedding that flag into the type system might improve readability. Conversely, having to look up what those wrappers are, learn about their Deref impls, and import/qualify them at every use, increases friction, so I'm not sure.

What I'd really like is [&[T]; N], but of course we can't have that until const generics (unless there's a modest upper bound on the number of channels? Supporting crazy professional DAW setups with boatloads of channels is cool and might preclude this, but there's always the dynamic API...).

[thavlik]

Has any additional conceptual progress been made on this? I'm looking to implement the functionality, so it make sense to do it as a PR.