search

tlambert03 / nd2

Full-featured nd2 (Nikon NIS Elements) file reader for python. Outputs to numpy, dask, and xarray. Exhaustive metadata extraction
https://tlambert03.github.io/nd2
BSD 3-Clause "New" or "Revised" License
54 stars 15 forks source link

Add __getstate__ and __setstate__ methods to allow pickling. Addresses #19 #20

Closed VolkerH closed 3 years ago

VolkerH commented 3 years ago

As discussed in #19, the dask arrays currently returned by ND2file.to_dask() cannot be pickled, which limits the use of these returned dask arrays.

The reason why the pickling fails is due to the cythonized _rdr. This PR adds a __getstate__ method that removes _rdr from the items to be pickled and a __setstate__ method that re-creates a _rdr from the path.

I can't build the python package as I don't have the C code for the Nikon SDK. So far I tested this by monkey-patching a pip-installed nd2 package with these two methods.

I am not sure whether the way I hanlde the .closed attribute is correct.

tlambert03 commented 3 years ago

thank you @VolkerH!

I can't build the python package as I don't have the C code for the Nikon SDK. So far I tested this by monkey-patching a pip-installed nd2 package with these two methods.

sorry about this. I think I'm going to just include the SDK here. The folks at laboratory imaging have given permission to distribute, and it would simplify a lot of things...

test fails are build-related (this is the first PR that's come outside of this repo)... so I'll deal with those too.

tlambert03 commented 3 years ago

@VolkerH could you merge main here? I tried to do it for you but keep running into issues

VolkerH commented 3 years ago

Hi @tlambert03 , done !

VolkerH commented 3 years ago

I can now build locally. Might add a short test, consisting of the following steps:

Is there somewhere I can fetch the nd2 files you are using for tests?

tlambert03 commented 3 years ago

thanks @VolkerH! Will merge now and add a test later. I need to revamp the testing suite to make at least a subset of it easier for contributors to use. but let me do that in another PR