tlambert03 / nd2

Full-featured nd2 (Nikon NIS Elements) file reader for python. Outputs to numpy, dask, and xarray. Exhaustive metadata extraction
https://tlambert03.github.io/nd2
BSD 3-Clause "New" or "Revised" License
53 stars 15 forks source link

Better dask wrapper that obeys NEP18 #26

Closed tlambert03 closed 2 years ago

tlambert03 commented 2 years ago

@VolkerH

Here's a solution that I think works well now. It's actually a subclass of dask.Array, and it passes all the tests. It's significantly faster than not using the subclass (bench below) so I'm going to leave it as the default, but now there's an opening_array parameter that you can set to False to get a pure dask.Array.

Also, I added logic to the _dask_block function that will re-open/close the file if it's closed. So, it will no longer fail to use opening_array = False ... even if the underlying resource has been closed. It will just be slower... (you can speed it up again by using your own context manager to re-open the file before computing on the array).

lemme know how it works for you, and if you have any suggestions/requests

codecov[bot] commented 2 years ago

Codecov Report

Merging #26 (72c062a) into main (c882be6) will increase coverage by 0.63%. The diff coverage is 97.56%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #26      +/-   ##
==========================================
+ Coverage   86.66%   87.29%   +0.63%     
==========================================
  Files          11       11              
  Lines        1117     1149      +32     
==========================================
+ Hits          968     1003      +35     
+ Misses        149      146       -3     
Impacted Files Coverage Δ
src/nd2/nd2file.py 91.82% <96.15%> (+2.03%) :arrow_up:
src/nd2/opening_dask_array.py 98.21% <98.21%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c882be6...72c062a. Read the comment docs.