search

HDFGroup / hsds

Cloud-native, service based access to HDF data
https://www.hdfgroup.org/solutions/hdf-kita/
Apache License 2.0
128 stars 52 forks source link

`relative paths or not valid` when trying to connect to an absolute path with "/" folder delimiters #125

Closed itsMeBrice closed 2 years ago

itsMeBrice commented 2 years ago

When trying to open a file with h5pyd.File the call returns relative paths or not valid when trying to connect with an absolute path (e.g. /a/b/c.h5 - the same call works fine when called with . as folder delimiters instead (e.g. a.b.c). The culprit seems to lie on lines 185-186 in _hl/files.py:

if domain.find('/') > 0: 
    raise IOError(400, "relative paths or not valid")

This check seems not to provide the intended functionality as it prohibits any opening of / sperated paths. The check should probably be: if domain.find('/.') > 0: or if domain.find('/..') > 0:

jreadey commented 2 years ago

Hey, There's a bit of legacy-related logic to this. Originally in the development of the REST API we had the idea to use DNS style paths. E.g. a domain could have posix-style of "/myfolder/myfile" or a DNS style: "myfile.myfolder". The idea was to somehow route requests to a HSDS instance like DNS names get assigned to IPs.

Anyway, that never came to fruition, and on the whole is rather confusing. Also, it was hard to support filename extensions like ".h5". HSDS itself still supports the DNS pattern, but I don't think h5pyd does.

So I checked in a h5pyd change that just verifies the first character of the domain name is a '/'. This means paths like "a.b.c" will raise the "relative paths are not valid" error.

Paths like '/myfolder/myfile.h5' should have worked before though. It's not clear to me why the absolute path would be rejected. Anyway, try out the last master branch update.

If anyone has a case to retain the DNS-style functionality, please respond here!

itsMeBrice commented 2 years ago

Hello, So DNS style paths will not be supported in the future? because with h5pyd < 0.8.4 DNS style paths worked just fine. Also I obiously filed this issue under the wrong project (hsds instead of h5pyd).

jreadey commented 2 years ago

Most of the HSDS integration tests use DNS paths, so HSDS support won't be going away anytime soon. We haven't been testing DNS paths with h5pyd though, so I wouldn't be surprised is something is broken there.

Is there some aspect of DNS style paths that is helpful for you?

itsMeBrice commented 2 years ago

Hello again, Finally got around to updating our hsds server to the newest master branch and i no longer encounter this error. Seems like it was fixed or just an user error. Thanks for the help!