Open rsignell-usgs opened 3 years ago
This is what @davidbrochart was working on in #28. That stalled because we did some internal refactoring which broke his approach. But maybe we can convince him to pick it up again? 😊
I'd be happy to work on this, but didn't have time so far :disappointed:
I wish we could argue that this is within scope for the Quantstack Zarr EOSS work! 😆
Me too! But not related to C++ or Zarr v3 unfortunately. If QuantStack could be involved in another Zarr project, I'd love to work on these things.
When I create a large rechunked dataset, I use rechunker on each time chunk, then append. The last time chunk currently fails because rechunker thinks the chunk plan is invalid (because only a single partial chunk is being written).
Here's a full example: https://nbviewer.jupyter.org/gist/rsignell-usgs/c0b87ed1fa5fc694e665fb789e8381bb