search

NeurodataWithoutBorders / lindi

Linked Data Interface (LINDI) - cloud-friendly access to NWB data
BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

Write to Zarr store using h5py-like interface #28

Closed magland closed 4 months ago

magland commented 5 months ago

This PR adds support for writing to Zarr stores using the h5py-like interface.

See https://github.com/NeurodataWithoutBorders/lindi/blob/write/tests/test_zarr_write.py

and

https://github.com/NeurodataWithoutBorders/lindi/blob/write/examples/example_create_zarr_nwb.py

codecov-commenter commented 5 months ago

Welcome to Codecov :tada:

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered :open_umbrella:

oruebel commented 5 months ago

Just to clarify, if I understand this correctly, lindi.LindiH5pyFile is here backed by a Zarr store, i.e., it is representing a Zarr file as if it where h5py and as such, pynwb.NWBHDF5IO is here actually writing to Zarr. I.e., in some sense this would be in a form a replacement forZarrIO in hdmf_zarr.

https://github.com/NeurodataWithoutBorders/lindi/blob/0f6814a0a7303ef4ce208cfde2d8980c112fe92d/examples/example_create_zarr_nwb.py#L16-L20

magland commented 5 months ago

Just to clarify, if I understand this correctly, lindi.LindiH5pyFile is here backed by a Zarr store, i.e., it is representing a Zarr file as if it where h5py and as such, pynwb.NWBHDF5IO is here actually writing to Zarr. I.e., in some sense this would be in a form a replacement forZarrIO in hdmf_zarr.

https://github.com/NeurodataWithoutBorders/lindi/blob/0f6814a0a7303ef4ce208cfde2d8980c112fe92d/examples/example_create_zarr_nwb.py#L16-L20

@oruebel Yes that's right.

rly commented 4 months ago

The suggested changes around np.float128/np.complex256 and the change of mode="a" to mode="r+" were necessary to make examples/example_create_zarr_nwb.py run on my machine.

rly commented 4 months ago

I did not do heavy testing of this PR, but I read through it all, and as far as I can tell, it makes sense and looks good. Nicely done!

HDMF supports a lot of weird edge cases. I'll see if I can think up more tests to what you already have here.

magland commented 4 months ago

I did not do heavy testing of this PR, but I read through it all, and as far as I can tell, it makes sense and looks good. Nicely done!

HDMF supports a lot of weird edge cases. I'll see if I can think up more tests to what you already have here.

Thanks Ryan!