ecmwf / eccodes-python

Python interface to the ecCodes GRIB/BUFR decoder/encoder
Apache License 2.0
113 stars 33 forks source link

Add support for `GRIB_TYPE_BYTES` to `codes_get_native_type` and `codes_get_array` #70

Closed gernotgeppert closed 1 year ago

gernotgeppert commented 1 year ago

Calling eccodes.codes_get_native_type for a key with type GRIB_TYPE_BYTES returns None. It should return bytes.

import os
os.environ['ECCODES_DEFINITION_PATH']='definitions.edzw-2.27.0-1'

import eccodes

with open('icon_grib_template') as f:
    gid = eccodes.codes_grib_new_from_file(f)
print(eccodes.codes_get_native_type(gid, 'uuidOfHGrid'))
eccodes.codes_release(gid)

# prints None

To extend grib_get_native_type, it seems to be sufficient to add a line here: https://github.com/ecmwf/eccodes-python/blob/bbd1a79812c37fab19ad740a6fc41597c1a59b73/gribapi/gribapi.py#L49

Subsequently, grib_get_array can be extended with a call to a new function grib_get_bytes_array.

eccodes 2.27.0 eccodes-python 1.5.0

definitions-edzw.zip icon_grib_template.zip

gernotgeppert commented 1 year ago

See https://github.com/ecmwf/eccodes-python/issues/71#issuecomment-1440617268 for a possible workaround using gribapi.lib.grib_get_native_type.

shahramn commented 1 year ago

I will add the extra type "bytes" but when it comes to decoding it, I will use the codes_get_string function. This seems a reasonable solution e.g.

elif ktype is str or ktype is bytes:
  result = grib_get_string(msgid, key)
gernotgeppert commented 1 year ago

Using grib_get_string in both cases seems like a good idea.

You could further consider the following to return an object with type bytes at decoding:

if kytpe is str:
    result = grib_get_string(msgid, key)
if ktype is bytes:
    result = grib_get_string(msgid, key).encode(ENC)

Otherwise, I'd suggest to add a note in the documentation that grib_get returns str for keys of type GRIB_TYPE_STRING and GRIB_TYPE_BYTES.

shahramn commented 1 year ago

This has been implemented now and will be in the next release. Many thanks