HDFGroup / h5pyd

h5py distributed - Python client library for HDF Rest API
Other
110 stars 39 forks source link

Support for writing attributes with H5S_NULL type #102

Closed jananzhu closed 3 years ago

jananzhu commented 3 years ago

Hi, I'm working with a dataset that contains an attribute with a H5S_NULL type, represented as h5py.Empty when accessed with h5py, and I'm wondering if h5pyd currently supports writing this attribute to datasets in HSDS.

Simply passing using h5py.Empty as the data parameter to attrs.create results in a TypeError: unknown object type. From the h5serv documentation for the attribute PUT operation, the REST API should support writing null attributes by specifying H5S_NULL as the shape, but I've been getting TypeErrors there as well when I try to use that as the shape parameter to attrs.create.

jreadey commented 3 years ago

Hey, I think I have a fix for this in the jreadey-master branch. Please try it out and let me know if that resolves the issue.

jananzhu commented 3 years ago

Hi, thanks for the reply! I tried running the following snippet with h5pyd built from the branch:

with h5pyd.File("/admin/test.h5", mode="w") as remote:
    remote.attrs.create("empty_attr", data=h5py.Empty)

and I'm still getting the error

  File ".../lib/python3.7/site-packages/h5pyd/_hl/attrs.py", line 258, in create
    type_json = getTypeItem(dtype)
  File ".../lib/python3.7/site-packages/h5pyd/_hl/h5type.py", line 330, in getTypeItem
    raise TypeError("unknown object type")
TypeError: unknown object type
jreadey commented 3 years ago

Hey, you'll need to pass a type to the Empty constructor like this:

  with h5pyd.File("/admin/test.h5", mode="w") as remote:
      remote.attrs.create("empty_attr", data=h5py.Empty('f'))

I think this is the case even for h5py

jreadey commented 3 years ago

Actually, there's another problem... h5py.Empty is a different class than h5pyd.Empty. So you'll need:

  with h5pyd.File("/admin/test.h5", mode="w") as remote:
    remote.attrs.create("empty_attr", data=h5pyd.Empty('f'))
jananzhu commented 3 years ago

Works perfectly now, thank you! Will leave this issue for you to close with the merge.

jreadey commented 3 years ago

This is in master and released to pypi as version 0.8.4.