188 enabled loading into the same volume from multiple zarr sources (resolving most of #78), and included code to ensure those sources have scale levels with matching dimensions. However, its implementation has an important limitation: matching scale levels also must have matching chunk sizes. This limitation wasn't imposed due to any difficulties with fetching zarrs with different chunking strategies - zarrita will happily accept a slice into an array with any chunking strategy and work out which chunks to fetch on its own. But our prefetching implementation assumed that it could safely apply the same chunk dimensions to every channel, which is not safe when different channels may come out of different source zarrs with different chunking strategies. So this PR allows us to deal with per-channel chunk dims when prefetching, removing the need for matching chunking strategies.
Changes
ChunkPrefetchIterator.ts: ChunkPrefetchIterator now accepts chunk sizes per source. The type of the corresponding field in the ChunkPrefetchIterator constructor was formerly TZYX ([number, number, number, number]), representing the number of chunks along those four axes. The type is now TCZYX<number>[], representing the number of chunks along these four axes per source, and including C (the number of channels in the given source) to match up sources with channels.
OmeZarrLoader.ts: now provides per-source dimensions.
ChunkPrefetchIterator.test.ts: updated to use the new dimensions type described above, and added a test to verify that per-source bounds are handled as specified.
zarr_utils/utils.ts: now that the above changes have been made, remove the restriction on matching chunking strategies.
Review time: moderate (~30min)
188 enabled loading into the same volume from multiple zarr sources (resolving most of #78), and included code to ensure those sources have scale levels with matching dimensions. However, its implementation has an important limitation: matching scale levels also must have matching chunk sizes. This limitation wasn't imposed due to any difficulties with fetching zarrs with different chunking strategies - zarrita will happily accept a slice into an array with any chunking strategy and work out which chunks to fetch on its own. But our prefetching implementation assumed that it could safely apply the same chunk dimensions to every channel, which is not safe when different channels may come out of different source zarrs with different chunking strategies. So this PR allows us to deal with per-channel chunk dims when prefetching, removing the need for matching chunking strategies.
Changes
ChunkPrefetchIterator
now accepts chunk sizes per source. The type of the corresponding field in theChunkPrefetchIterator
constructor was formerlyTZYX
([number, number, number, number]
), representing the number of chunks along those four axes. The type is nowTCZYX<number>[]
, representing the number of chunks along these four axes per source, and including C (the number of channels in the given source) to match up sources with channels.