pangeo-data / rechunker

Disk-to-disk chunk transformation for chunked arrays.
https://rechunker.readthedocs.io/
MIT License
164 stars 25 forks source link

added write_empty_chunks to ZARR_OPTIONS #119

Closed norlandrhagen closed 2 years ago

norlandrhagen commented 2 years ago

Adds write_empty_chunks to ZARR_OPTIONS. This should allow the write_empty_chunks to be passed through to zarr.

Similar functionality to this xarray PR: https://github.com/pydata/xarray/pull/6348

Closes #94

norlandrhagen commented 2 years ago

cc @andersy005

andersy005 commented 2 years ago

The write_emtpy_chunks option is available starting with xarray>2022.3 and zarr>=2.11. However, rechunker doesn't enforce lower bounds on zarr and xarray versions. should we add a check for this? cc @rabernat

rabernat commented 2 years ago

Thanks for this @norlandrhagen!

It's fine with me if we depend on a minimum zarr version.

codecov[bot] commented 2 years ago

Codecov Report

Merging #119 (0ad5886) into master (b55c3b6) will decrease coverage by 0.39%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #119      +/-   ##
==========================================
- Coverage   96.42%   96.02%   -0.40%     
==========================================
  Files          11       11              
  Lines         503      503              
  Branches      112      112              
==========================================
- Hits          485      483       -2     
- Misses         11       13       +2     
  Partials        7        7              
Impacted Files Coverage Δ
rechunker/api.py 98.04% <ø> (ø)
rechunker/compat.py 77.77% <0.00%> (-22.23%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b55c3b6...0ad5886. Read the comment docs.

norlandrhagen commented 2 years ago

Added minimum versions for dask and zarr in setup.py. LMK if this is the wrong place!

rabernat commented 2 years ago

Sorry for letting this hang for so long! 🤦

If we add these minimum versions, it looks like we also have to drop python 3.7 support. That's fine with me.

rabernat commented 2 years ago

I will merge this and then make a new PR for dropping python 3.7.