Closed manzt closed 4 years ago
I've had some discussion with Heath at Vanderbilt about creating an identical zarr store which contains the mz channels in a more contiguous ordering. The tradeoff here is that we add computation and increase storage, but it will afford better performance when trying to make selections along the mz axis of the data.
Use case: we fetch the x,y optimized store for "images" at certain mz channels, and then use x,y selection from the screen to fetch the mz ratios for a selection and show a spectrum view. Heath noted that the biggest commercial software (scils.de) for IMS does this type of optimization.
Fixes #70 and adds thoughtful chunking.
Under the hood, n-dimensional arrays are stored as contiguous buffers. Elements at particular indices can be accessed in each dimension of an array by using strides and offsets:
Thus, the ordering of dimensions is important depending on what dimensions need to be accessed quickly. For rendering 2D layers (i.e. a particular mz channel), it makes sense to chunk our arrays so that we can quickly access a full channel sequence on the client.