search

HDFGroup / hdf5-json

Specification and tools for representing HDF5 in JSON
https://hdf5-json.readthedocs.io
Other
72 stars 25 forks source link

h5tojson fails with Type Error #74

Closed GitLearner25 closed 1 year ago

GitLearner25 commented 2 years ago

Hi,

I am currently trying to convert an HDF5 file to JSON but running into an issue with types as below:

File "/home/hdfs/envs/parser_env/lib/python3.8/site-packages/h5json-1.1.3-py3.8.egg/h5json/h5tojson/h5tojson.py", line 244, in main dumper.dumpFile() File "/home/hdfs/envs/parser_env/lib/python3.8/site-packages/h5json-1.1.3-py3.8.egg/h5json/h5tojson/h5tojson.py", line 204, in dumpFile print(json.dumps(self.json, sort_keys=True, indent=4)) File "/usr/local/lib/python3.8/json/init.py", line 234, in dumps return cls( File "/usr/local/lib/python3.8/json/encoder.py", line 201, in encode chunks = list(chunks) File "/usr/local/lib/python3.8/json/encoder.py", line 431, in _iterencode yield from _iterencode_dict(o, _current_indent_level) File "/usr/local/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict yield from chunks File "/usr/local/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict yield from chunks File "/usr/local/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict yield from chunks File "/usr/local/lib/python3.8/json/encoder.py", line 325, in _iterencode_list yield from chunks File "/usr/local/lib/python3.8/json/encoder.py", line 438, in _iterencode o = _default(o) File "/usr/local/lib/python3.8/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.class.name} ' TypeError: Object of type bytes is not JSON serializable

I understand that this is because JSON expect strings and not bytes but I am not sure what is causing this error in the HDF5 file to see if I can convert it into string before the JSON conversion. Do you know any workarounds for this?

jreadey commented 2 years ago

It's hard to say what the problem could be just based on the stack trace. Is the HDF5 file you are trying to convert something you can share?

ajelenak commented 2 years ago

@GitLearner25 I would suggest installing the h5json package from this repository and trying again your file. There have been a number of fixes from the last release. The instructions how to do this are here. Or you can share the file...

GitLearner25 commented 2 years ago

Thanks for the quick response! I am unfortunately not able to share the HDF5 file since it contains some confidential data but I will try installing the h5json package from this repository and try again.

ajelenak commented 2 years ago

@GitLearner25 Do you still have this problem?

GitLearner25 commented 1 year ago

Hi Ajelenak,

Apologies for the late response - I do not. Pulling the package from the repository helped resolve the issue, thanks!

I did run into another issue where I am trying to convert a relatively large H5 file to JSON but it fails with the message "Killed". I can open a different issue for that if thats more appropriate but any thoughts on what might be causing it? I was thinking it was possibly a space issue on my machine.

ajelenak commented 1 year ago

Yes, it should be a new issue.

I suggest using the -D option to avoid dumping dataset values into JSON.

GitLearner25 commented 1 year ago

I will try it, thank you.