search

HDFGroup / hsds

Cloud-native, service based access to HDF data
https://www.hdfgroup.org/solutions/hdf-kita/
Apache License 2.0
128 stars 52 forks source link

Warn or error for long socket directory paths #129

Closed ajelenak closed 2 years ago

ajelenak commented 2 years ago

When trying to run HSDS in the nodocker mode, socket directory paths above certain number of characters can break low-level socket communication code. I encountered this problem when setting up nodocker HSDS in a pytest fixture on a macOS computer. Temporary directories on macOS have long paths and prevent communication with HSDS if used as socket directories.

Two reported errors are: AF_UNIX path too long or UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long). For the first error, the limit seems to be somewhere around 104-108 characters, while for the second 64 characters.

I suggest that HSDS when started in nodocker mode warns or raises an exception if socket directory path is longer than 64 characters.

jreadey commented 2 years ago

I've put in a check here: https://github.com/HDFGroup/hsds/commit/f7d8f0cccf3379b50a0299f8bb48b84debff0808. It's unfortunate that tempfile package on MacOS returns such long directory names. Currently when a socket_dir is not specified hsds_app is using /tmp/hs1234abcd/ as the directory name where "1234abcd" is a random hex string.

ajelenak commented 2 years ago

It is funny there are still these limits from ye olde days of computing. 😁

Thanks!

jreadey commented 2 years ago

Yes, something to do with max hostname length - see https://bugs.python.org/issue32958. Why this should matter when you are using sockets, I don't know.