Open MarkKremer opened 11 months ago
Speaking to the speaker/Oto 2 channel problem: in addition to split and merge, there could be a, let's call it, MapChannels where you could specify how the n channels gets merged into 1 or 2 channels.
Something like
MapChannels(leftRightMapper{Left: []channel{channels[0], channels[2]}, Right: []channel{channels[1], channels[3]}})
Something like this could be a stereo to mono mapper
// Maps mono audio to stereo output
MapChannels(leftRightMapper{Left: []channel{channels[0]}, Right: []channel{channels[0]}})
For the proposal as a whole, I think it makes sense to have the channel information near the samples (and thus allow samples to have n channels). I've had a project where having the methods would've alone made it much easier to think about (I was mapping audio sent across the network to beep).
I wonder then if it would be worth having those that only support 1 or two channels, to have a special case streamer such that it's not possible to feed a 6 channel streamer into an speaker (for example). I feel that that could breed confusion and annoy to find bugs.
Current state
I would say that the 2 most important types in Beep are the following:
Streamer
allows us to define operations on samples. Using the composite pattern it is possible to combine operations to create more complex operations.Format
, besides storing the format information, is used to encode/decode samples into different representations.These types are very powerful and can be used to do a lot of things with very little. However, there are some details about them that make me wonder if something better is possible:
Format
specifies the number of channels, within the interface of theStreamer
, the number of channels is hardcoded to2
. I suspect this is done to keep the library simple and this is an important consideration. Please try to keep this in mind when reading the rest of this proposal.Precision
andSampleRate
are mostly used at endpoints: when encoding/decoding a file and when using an in-memory buffer (which is similar to a WAV file). In addition,SampleRate
can be used when resampling samples.The number of channels seems like it's an inherent property of the samples while the
Format
is only used at specific parts of the application. It is metadata that is exposed when decoding a file format, or it can passed as configuration to encode audio.Format
is however, never directly used byStreamer
s and is completely separate from the composite pattern that is core to Beep.Proposal
Move
NumChannels
toSamples
.The samples are stored in an interleaved format in a 1D slice. We lose the syntactic sugar of 2D slices which I solved by using methods (BOOO!). I think the benefits could very well outweigh the drawbacks but I would like to invite you to think about the developer experience for the users of Beep when, say, they want to implement a custom
Streamer
.For reference, this is what the types will look like (approximately):
In this scenario,
Format
can be used to format individual samples still. However, it doesn't deal with framing samples of channels together.What do we gain?
One obvious benefit is that the number of channels isn't constant anymore:
Doppler
effect the audio is converted to 2 or more different channels for the speakers to play and your brain to interpret.Furthermore: operations on channels.
Operations on channels
Because the channel count is stored in the
Samples
struct,Streamer
operations that act on those channels become a possibility. This gives the user better control of what they want to do:I suspect the implementation of
SplitChannels()
andMergeChannels()
will be a bit more complex than it may look at first. But I think it is doable.Cons
Streamer
becomes slightly more complex in some way.Streamer
must support different values forNumChannels
or return an error if the channel count is unsupported.Streamer
you have to the required number of channels. However, the tools to do so will be available to you (see previous code snippet).