search

ARM-DOE / pyart

The Python-ARM Radar Toolkit. A data model driven interactive toolkit for working with weather radar data.
https://arm-doe.github.io/pyart/
Other
501 stars 264 forks source link

Handling missing metadata fields in pyart.aux_io.read_odim_h5() reader #1599

Open jorahu opened 2 months ago

jorahu commented 2 months ago

Description

Importing h5 file with pyart.aux_io.read_odim_h5() could not complete due to missing "version" or "source" field in file metadata. The problem is with the lines 169 and 170 in https://github.com/ARM-DOE/pyart/blob/main/pyart/aux_io/odim_h5.py There should be a check like there is for example lines 177 and 178.

What I Did

First, line 164 should be deleted as it is duplicated at line 170.

The lines 168-170 could be changed as follows:

h_what = hfile["what"].attrs 
if "version" in h_what:
    metadata["version"] = _to_str(h_what["version"])
if "source" in h_what:
    metadata["source"] = _to_str(h_what["source"])

I am not sure how this complies with ODIM structure, but I tested it with 2 completely different files (from different parts of the world), one had missing "source", other had missing "version" for unknown reasons. One missing (potentially unimportant) metadata field should not stop the whole reading process.

mgrover1 commented 2 months ago

@jorahu - any interest in submitting a PR with the fix here? Thanks for highlighting the current limitation.

Would you be interested in supplying those sample files so we can add that to the test suite?

jorahu commented 2 months ago

I was looking into making the pull request before raising the issue, but unfortunately I am too noob with github to do that. Sorry :(

One of the test files got another error (but the first error was fixed with this previous code), I also checked with wradlib importer, that's why I knew about the missing fields. But I used this file just to test this issue, don't have time to track down the other problem at the moment. And the initial file I tried was sent to me for testing, so I need to ask if it's okay to share it.

jorahu commented 2 months ago

Yeah, see... can't even comment without messing it up. I'm pretty new to github. Apologies.

mgrover1 commented 2 months ago

No worries! If you could share some of the files to test, that would be ideal, to make sure that the fix works generically!

Feel free to share it with my email, mgrover@anl.gov