10m wind output is all zero

[jwolff-ncar]

When running EMC_post via the SRW App (using the postxconfig-NT-fv3lam.txt flat file), the post output file creates 10m wind output but it is all zero.

For example: wgrib2 -V -d 1395 rrfs.t00z.bgrd3df006.tm00.grib2 1395:5308958455:vt=2019041506:10 m above ground:5-6 hour max fcst:WIND Wind Speed [m/s]: ndata=10777921:undef=932171:mean=0:min=0:max=0 grid_template=1:winds(N/S): rotated lat-lon grid:(4081 x 2641) units 1e-06 input WE:SN output WE:SN res 48 lat -33.000000 to 33.000000 by 0.025000 lon 309.000000 to 51.000000 by 0.025000 #points=10777921 south pole lat=-42.000000 lon=248.000000 angle of rot=0.000000

A short-term solution to allow MET to derive the field internally from the U/V-components would be to turn off this field in the flat file while a bug fix is implemented.

[fossell]

@jwolff-ncar - What version/branch of UPP is being used for this? Is this from bug fix release testing or separate development testing using develop branch?

[jwolff-ncar]

@fossell There are a few different versions that this has recently been discovered in: hash = 9fa1e088 and hash = a49af05 This was identified through separate testing not related to the bug fix release.

[fossell]

@jwolff-ncar - It looks like this field is possibly just a "pass-through" variable meaning it is read from the model output and then directly output into the grib2 file. Can you see if there are valid data in the variable "spd10max" of your SRW output? If yes, then we'll keep digging here.
@hertneky - Can you confirm this as a pass-through variable too? Just want to make sure I'm not overlooking something.

[jwolff-ncar]

Good to know, @fossell. I took a look at our output and spd10max is not in there (it is not included the diag_table.FV3_RRFS_v1alpha). In fact it is not included in most of the diag_tables included in the SRW App (it is only in diag_table.FV3_GFS_2017_gfdlmp_regional) That would explain it. We will have to decide as a team how to proceed. Thanks for your help.

[hertneky]

@fossell @jwolff-ncar I also see it as a pass through variable, reading the field 'spd10max' from the model output as Kate mentioned.

[fossell]

@jwolff-ncar - Ok great, well I mean not great that it's not in your model output, but I'm glad there isn't a bug or problem! Let's know if you encounter problems trying to output it in the future, happy to help. Closing this now.

[jwolff-ncar]

@fossell Actually, if it is not in the model output why does it turn to 0 coming out of post? Shouldn't it not be included in the post output then?

[fossell]

@jwolff-ncar - If it is requested in the postxconfig file, UPP will attempt to output that field. It is not smart enough to dynamically know what fields from model are required to produce the requested output fields and error out if there is missing data. But that sort of mapping has been discussed as a benefit, it's heavier lift than we have resources for at the moment though. Typically a field is initialized to zeros or undefined and then filled/computed. In this case it looks like it is just set to all zero.

[jwolff-ncar]

Ok. It is a bit unfortunate it is filled with zeros since that is a plausible value! Thanks again for your help!

[fossell]

@jwolff-ncar - Ah ok, I see what you're saying now. Apologies, I think I misunderstood the concern before. Agreed that is a little misleading, and actually I'm not sure why this is a case. There should be a uniform assignment of missing values if a variable is not present in the model output, this value is typically 9.9e10. I can reopen this issue and we can try to figure out what's going on here for this field.

[fossell]

@WenMeng-NOAA - Do you know what groups at EMC use the postxconfig-NT-fv3lam.txt for product generation? Is it used in operations?

The SRW App team has come across an issue with one of the fields requested (MAX_WIND_ON_SPEC_HGT_LVL_ABOVE_GRND_10m; UPP ID 422) that is not in the model output (fv3lam output field is called "spd10max") resulting in a grib2 record of all zeros. This is undesirable as downstream plotting utilities (METplus) still plot the field even though it shouldn't exist. Using undefined values is also undesirable because downstream plotting utilities will break. The preference is to remove this field from the postxconfig file, and permanently if possible. But of course that depends on who else (if anyone) outputs this product using postxconfig-NT-fv3lam.txt. They've determined that this field is only requested for this configuration: diag_table.FV3_GFS_2017_gfdlmp_regional. Is that still used by any group at EMC? Is there another contact person I should ask about FV3LAM products?

Perhaps a larger question for another day is should all configurations of fv3lam output the same products? Or should each configuration have it's own specific product list (postxconfig file)?

I've provided the SRW App team with a modified postxconfig file with that field removed, but we'd like to explore the option of modifying it in the develop branch as well.

[WenMeng-NOAA]

@fossell The postxconfig-NT-fv3lam.txt has been used for RRFS operational development. I looked into the UPP regression tests for RRFS. It appears to me that the variable "spd10max" is in the RRFS model output. So the UPP output in GRIB2 has non-zero values for field 10m wind. You might contact with EMC RRFS team regarding the SRW app issue: jacob.carley@noaa.gov, eric.rogers@noaa.gov, benjamin.blake@noaa.gov

[fossell]

@WenMeng-NOAA - Thanks for the info, very helpful. We'll touch base with RRFS and see what next steps are.

[fossell]

@jwolff-ncar - Per email conversation it sounds like this is mostly resolved? The spd10max will be added to the diag tables for your SRW App configuration eventually, correct? And there is a solution in METplus to compute the instantaneous wind at 10m and ignore the current missing max wind field. Please let me know if there is anything you need UPP to do differently now or in the future. Otherwise I'll close this issue. Thanks!

[jwolff-ncar]

@fossell I think there is still a need to understand why, when there is no spd10max field in the model output, UPP still outputs a field of zeros when it is requested in the flat file. It should be undefined or bad data, I would think.

[fossell]

@fossell I think there is still a need to understand why, when there is no spd10max field in the model output, UPP still outputs a field of zeros when it is requested in the flat file. It should be undefined or bad data, I would think.

@jwolff-ncar, Yes we can look into that and make sure UPP is behaving as we expect and not giving these misleading 0s. I just want to make sure I understand what problems we have and are trying to solve... if that field is output as undefined or bad data as expected instead of zeros, then are you good to go with your configuration that doesn't output spd10max in model history files? Or does that still cause problems with METplus?

[fossell]

@WenMeng-NOAA - We've have some email exchanges with @ericaligo-NOAA regarding this hourly max wind speed at 10m AGL field (UPP ID 422). As described before, this is model variable spd10max that is simply read in from model history files in the INITPOST_NETCDF routine, passed through UPP and output in the grib file. In this case, that spd10max is not present in the model history file, so UPP fills the array with spval/undefined values, as expected:

! max hourly 10m agl wind speed
    VarName='spd10max'
    call read_netcdf_2d_scatter(me,ncid2d,1,im,jm,jsta,jsta_2l &
     ,jend_2u,MPI_COMM_COMP,icnt,idsp,spval,VarName,wspd10max)
     if(debugprint)print*,'sample ',VarName,' = ',wspd10max(isa,jsa)`

In a test log file, output indicates as expected that the field is not found and filled with missing values.

...
 spd10max             not found -Assigned missing values
...

However, when inspecting the grib2 file, we see that the record has all values of 0:

%wgrib2 -V NATLEV.036 -d 1303
1303:15793692:vt=2019061612:10 m above ground:35-36 hour max fcst:WIND Wind Speed [m/s]:
    ndata=23115:undef=23115:mean=0:min=0:max=0
    grid_template=30:winds(N/S):
    Lambert Conformal: (201 x 115) input WE:SN output WE:SN res 0
    Lat1 22.959871 Lon1 238.655396 LoV 262.500000
    LatD 38.520000 Latin1 38.520000 Latin2 38.520000
    LatSP 0.000000 LonSP 0.000000
    North Pole (201 x 115) Dx 25000.000000 m Dy 25000.000000 m mode 0

Both @ericaligo-NOAA and I have inspected the code and cannot find a place where this would be assigned zeros. So we are wondering if you might have an idea of why this is happening? Is there a way to have underfined values in the grib2 file?

[fossell]

I'm beginning to wonder if this is a grib utility thing? As I look at that wgrib2 output example above, there are 23,115 grid points that are undefined. Look at the second line and undef=23115 So wgrib2 is in fact finding that each grid point is an undefined value. Perhaps the mean/min/max=0 is a default for wgrib2 or something and not actually reflective of the values on the grid? I'm not sure how METplus uses wgrib2's undef info.

[fossell]

And actually, using wgrib2 to print out a sample value at i,j:

%wgrib2 NATLEV.036 -d 1303 -s -ij 50 50
1303:15793692:d=2019061500:WIND:10 m above ground:35-36 hour max fcst::val=9.999e+20

the value appears to be undefined, e.g. 9.999e+20

[WenMeng-NOAA]

I agree with Kate that might be the way of wgrib2 utility thing to dump a field with all undefined as min/max/mean as 0. You might check if a field is all undefined via comparing "ndata" and "undefine".

[fossell]

@WenMeng-NOAA - Just to wrap this up and so you're aware of the findings, Jamie and I were able to trace the problem to using an older verision of UPP from December 2020. In that version, the fields were not filled properly with missing values, so in the output some grid points appeared as zeros. We have confirmed this issue has since been fixed in develop branch. And to be clear, there is no issue with the min/max/mean values of zero from wgrib2, that's just a wgrib2 thing. The real problem w as the output grib files had actual zero values at grid points. But again, we've confirmed this has already been fixed in develop branch. Closing this issue now.

[WenMeng-NOAA]

@fossell Thanks for confirming. Sometimes, decoding GRIB2 data via wgrib2 is trivial.