HDFGroup / h5pyd

h5py distributed - Python client library for HDF Rest API
Other
111 stars 39 forks source link

Selection consistency between distributed and local h5py? #27

Closed vjcitn closed 7 years ago

vjcitn commented 7 years ago

Working with the tall data distributed with h5pyd, a simple selection generates invalid point argument error; below, the same operation succeeds with local hdf5 resource. Is there a reference on essential discrepancies between the two approaches?

%vjcair> python Python 2.7.12 (default, Nov 17 2016, 17:26:31) [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import h5pyd as h5py f = h5py.File("tall.data.hdfgroup.org", "r", endpoint="https://data.hdfgroup.org:7258") g2 = f['g2'] dset22 = g2['dset2.2'] dset22[[1,2]] Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/site-packages/h5pyd-0.1.0-py2.7.egg/h5pyd/_hl/dataset.py", line 664, in getitem raise ValueError("invalid point argument") ValueError: invalid point argument h5py.version.version '0.0.1'

%vjcair> python Python 2.7.12 (default, Nov 17 2016, 17:26:31) [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import h5py f = h5py.File("tall.h5") g2 = f['g2'] dset22 = g2['dset2.2'] dset22[[1,2]] array([[ 0. , 0.2 , 0.40000001, 0.60000002, 0.80000001], [ 0. , 0.30000001, 0.60000002, 0.89999998, 1.20000005]], dtype=float32) h5py.version.version '2.7.0'

jreadey commented 7 years ago

I've checked in a fix for the point selection problem: https://github.com/HDFGroup/h5pyd/commit/db6522432e70fdc12c18a69cd598b8f62ee8e9bc. Let me know if that works for you.

In general the goal is for h5pyd to be completely compatible with the h5py high-level api.

There are some features that are not yet supported in h5serv, hence won't work for h5pyd. e.g.:

And some features that work in h5serv, but have not yet been implemented in h5pyd, like region references.

And some things that don't work just because they are bugs!
Please file issues as you run across incompatibilities, and we'll address as soon as we can.

jreadey commented 7 years ago

Oops, looks like there is a problem with the travis build: https://travis-ci.org/HDFGroup/h5pyd/jobs/219155977. I'll look into this.

jreadey commented 7 years ago

Travis build is passing - closing.