search

pangeo-data / rechunker

Disk-to-disk chunk transformation for chunked arrays.
https://rechunker.readthedocs.io/
MIT License
162 stars 25 forks source link

Modify algorithm to try to reach chunk upper bound #116

Closed rabernat closed 2 years ago

rabernat commented 2 years ago

Fixes #115

This modifies the algorithm in a very slight way. For each axis, it attempts to maximize the chunk size up to the full array shape or the specific chunk shape limit (for example, as determined by target_chunks). Previously it would just expand the chunks by an integer multiple of the original chunks.

I also added some additional tests to make sure we do not produce situations that would require write locking.

codecov[bot] commented 2 years ago

Codecov Report

Merging #116 (9a40b01) into master (cbf969c) will increase coverage by 0.09%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #116      +/-   ##
==========================================
+ Coverage   96.32%   96.42%   +0.09%     
==========================================
  Files          11       11              
  Lines         490      503      +13     
  Branches      112      112              
==========================================
+ Hits          472      485      +13     
  Misses         11       11              
  Partials        7        7
Impacted Files Coverage Δ
rechunker/algorithm.py 85.71% <100.00%> (+3.25%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update cbf969c...9a40b01. Read the comment docs.

rabernat commented 2 years ago

The linting check is broken. We should switch to pre-commit.ci.