chiaral
commented
6 years ago After digging in the PyNIO page I found the GRIB2 options, and a description of how the names are created. Apologies for not having gone through it all before.
GRIB2 data variable name encoding (Note: examples show intermediate steps in the formation of the name)
if production status is TIGGE test or operational and matches entry in TIGGE table:
<parameter_short_name> (ex: t)
else if entry matching product discipline, parameter category, and parameter number is found:
<parameter_short_name> (ex: TMP)
else:
VAR_<product_discipline_number>_<parameter_category_number>_<parameter_number> (ex: VAR_3_0_9)
_P<product_definition_template_number> (ex: TMP_P0)
if single level type:
_L<level_type_number> (ex: TMP_P0_L103)
else if two levels of the same type:
_2L<level_type_number> (ex: TMP_P0_2L106)
else if two levels of different types:
_2L<_first_level_type_number>_<second_level_type_number> (ex: LCLD_P0_2L212_213)
if grid type is supported (fully or partially):
_G<grid_abbreviation><grid_number> (ex: UGRD_P0_L108_GLC0)
else:
_G<grid_number> (ex: UGRD_P0_2L104_G0)
if not statistically processed variable and not duplicate name the name is complete at this point.
if statistically-processed variable and constant statistical processing duration:
if statistical processing type is defined:
_<statistical_processing_type_abbreviation><statistical_processing_duration><duration_units> (ex: APCP_P8_L1_GLL0_acc3h)
else
_<statistical_processing_duration><duration_units> (ex: TMAX_P8_L103_GCA0_6h)
else if statistically-processed variable and variable-duration processing always begins at initial time:
_<statistical_processing_type_abbreviation> (ex: ssr_P11_GCA0_acc)
if variable name is duplicate of existing variable name (this should not normally occur):
_n (where n begins with 1 for first duplicate) (ex: TMAX_P8_L103_GCA0_6h_1)You also note on the same page linked above "Generally speaking all of the applicable characteristics must be the same for two records to be considered part of the same variable."
Is there a way/option to limit it to, for example, TMAX?
I am very new to GRIB2, and I usually extracted data using WGRIB2, or pygrib, using shortname to access the variable i am interested in. I am just curious to know why the choice to add these attributes (
Do you think this is a feature you would be interested in developing (meaning, limit the name to the variable name to just, i.e., parameter_short_name)?
That would make the usage of PyNIO within open_mfdataset() flawless, whereas now I need to write ad-hoc preprocessing function to change the variable name constructed by PyNIO.
thanks!
david-ian-brown
NicWayand
I posted this on SO, but I think it belongs here as well.
Below an edited version:
I am using the data found here (Note, these are rotating files of forecast data, so the actual date will change as time goes by, you might need to update the date in my example in a few days)
which is read as a dataset with some coordinates and variables
and many other variables... i am not copying here...
my struggle right now is to understand how these variable names (i.e. PLI_P0_2L108_GLL0) are generated.
If I run directly Nio:
I get:
etc...
but the names of the variables are already there. So it seems like these names are defined at Nio level (I think).
However, when I inspect the data with wgrib2 I have the following:
The variable names are PRES, HGT, ABSV, VGRD, ACPCP, and so on.
The problem I have is that I am trying to run an open_mfdataset in xarray to concatenate along multiple dimensions (in my case forecast start time and forecast lead time) thousands of these little files, unfortunately the variable names (of what should be the same quantity, i.e. convective precipitation, which in the grib file is simply named ACPC) generated by xarray/PyNIO change throughout the dataset going from ACPCP_P8_L1_GLL0_acc, to ACPCP_P8_L1_GLL0_acc6h, to ACPCP_P0_L1_GLL0, making things complicated.
I tried to overwrite the variable names with some preprocessing function, but I wanted to understand the rationale behind those names, to be sure that I am doing it correctly.
So, where does the part of the variable name, like "_P0_L6_GLL0", attached to the grib variable name come from? is there a way to control that?