Open jrevels opened 3 years ago
https://github.com/JuliaAudio/SampledSignals.jl also deals with multichannel signals and implements array type methods, so could be worth a look to see if we want to borrow any of that design.
relevant to this topic - it's worth noting that by this point, the OSS ecosystem that emerged from the geoscience space (xarray + zarr + kerchunk) basically solves most of the same problems as Onda's sample data manipulation functionality, but in a drastically more generalized domain-agnostic fashion (n-dimensional labeled-dimension array storage, pluggable file formats/codecs, pluggable storage systems)
If these tools had already existed way back in early 2019, Onda.jl wouldn't have consisted of anything more than the signal/annotation schemas themselves
This is probably an indication that the direction outlined by this issue is the right eventual path in theory
a secret window into the mysterious caverns of the Beacon Slack:
It always feels like there's lots of overlap between
Samples
and more generic named-array packages like AxisKeys/AxisArrays/etc. Would it be a good idea to refactor Samples to be a wrapper around AxisKeys, or define convenience constructors for translating between them, or replaceSamples
entirely, or something else altogether that allows Onda (and downstream callers) to more easily reuse/interface the more generic functionality provided by AxisKeys?It seems like answering this question is a matter of clearly stating the "responsibility" of the
Samples
API layer and possibly formulating our goals around that.To that end, it might be useful to start by roughly describing the manner in which Onda's functionality is currently layered:
AbstractLPCMFormat
API)Samples
API.The current core responsibilities of the
Samples
API layer are...load
/store
unit" for Onda-formatted sample data (in practice - associatesSampleInfo
with a corresponding sample data matrix)encode
/decode
functionality. "Why here instead of theAbstractLPCMFormat
layer?", you might ask. The answer is that theSamples
layer is the "lowermost" layer at whichencode
/decode
can be implemented generically w/o knowing anything about the target/source serialization format.SampleInfo
with a data matrix, like specialized time span/channel indexing.Given all of these points in combination, it seems like we can at least say we won't be able to get rid of
Samples
fully, unless a) we feel like everything we'd ever want out of point 3 could be achieved already with a more generic array type and b) we'd feel fine forcing callers to pass arounddata
andinfo
into most API functions as separate arguments (and having callers keep track on their own of whether or notdata
is encoded/decoded).So, assuming we wouldn't don't get rid of
Samples
entirely, how could we use these other packages to makeSamples
better? It seems like the primary annoyance that they might help with is thatSamples
not anAbstractMatrix
despite point 3. It's like telling callers "yeah,Samples
isn't anAbstractMatrix
, but here, if you want special matrix-y indexing features, wrap yourAbstractMatrix
in it!" Sounds weirdly inconvenient, but the reasoning kind of makes compositional sense - if you DID implement it as a fullAbstractMatrix
, then callers would still probably have to assume (outside a few special cases that are already covered) that mostAbstractArray
operations would causeSamples
inputs/outputs to be unwrapped anyway (i.e. the transform on theSamples
data
would not have a corresponding sensible transform on itsSamplesInfo
). Thus, we just keep the distinction very clear/explicit, forcing callers to unwrap/rewrap themselves, so that they don't land in weird situations where they accidentally loseSamplesInfo
along the way in an unexpected manner.Does all of this imply that Onda should just provide explicit constructors to go between
KeyedArray
andSamples
and be done with it? That seems like an easy enough thing to do. This brainstorming does make me want to play around with aSamples
-less API (which would be beholden toa)
andb)
mentioned above), just to see how far it can go...