HDFGroup / h5pyd

h5py distributed - Python client library for HDF Rest API
Other
110 stars 39 forks source link

use surrogateescaping for non-utf8 encodable attribute values in hslo… #138

Closed jreadey closed 1 year ago

jreadey commented 1 year ago

HSDS isn't able to store non-utf8 encodable attributes these values need to be transmitted and store as JSON. This update will convert any such value using data.encode("utf-8", errors="surrogateescaping"). The value with get stored with \U escape codes. On reading, the origin byte value will be returned.