search

NOAA-EMC / NCEPLIBS-g2

Utilities for coding/decoding GRIB2 messages.
Other
6 stars 15 forks source link

parameters with duplicate g1 numbers #311

Closed edwardhartnett closed 4 weeks ago

edwardhartnett commented 2 years ago

We have:

 data paramlist(1016) /gribparam(129, 240, 0, 16, 3, 'RETOP')/
  data paramlist(1017) /gribparam(2, 234, 1, 0, 5, 'BGRUN')/
  data paramlist(1018) /gribparam(2, 235, 1, 0, 6, 'SSRUN')/
  ! Added 06/12/2020
  data paramlist(1019) /gribparam(130, 160, 2, 3, 5, 'SOILL')/

These are duplicate grib1 numbers. What's up?

BoiVuong-NOAA commented 2 years ago

I do not see any duplicate grib1 number data paramlist(1016) /gribparam(129, 240, 0, 16, 3, 'RETOP')/ data paramlist(1017) /gribparam(2, 234, 1, 0, 5, 'BGRUN')/ data paramlist(1018) /gribparam(2, 235, 1, 0, 6, 'SSRUN')/ ! Added 06/12/2020 data paramlist(1019) /gribparam(130, 160, 2, 3, 5, 'SOILL')/

BoiVuong-NOAA commented 2 years ago

The reason for these duplicate grib1 numbers. Because some developers used it in local define table and WMO standard) WMO approved) in grib2 table. Example: SSRUN has entry in two tables: 1,0,6 (WMO approved) and 1,0,193 (user local define)

edwardhartnett commented 2 years ago

I should have been more clear. These GRIB1 numbers are duplicated at other places in the file.

data paramlist(1016) /gribparam(129, 240, 0, 16, 3, 'RETOP')/
 data paramlist(1017) /gribparam(2, 234, 1, 0, 5, 'BGRUN')/
 data paramlist(1018) /gribparam(2, 235, 1, 0, 6, 'SSRUN')/
 ! Added 06/12/2020
 data paramlist(1019) /gribparam(130, 160, 2, 3, 5, 'SOILL')/

For example, the first set of numbers, 129, 240, also appears here:

data paramlist(704) /gribparam(129, 240, 0, 16, 197, 'RETOP')/

edwardhartnett commented 2 years ago

I should have been more clear. These GRIB1 numbers are duplicated at other places in the file.

data paramlist(1016) /gribparam(129, 240, 0, 16, 3, 'RETOP')/
 data paramlist(1017) /gribparam(2, 234, 1, 0, 5, 'BGRUN')/
 data paramlist(1018) /gribparam(2, 235, 1, 0, 6, 'SSRUN')/
 ! Added 06/12/2020
 data paramlist(1019) /gribparam(130, 160, 2, 3, 5, 'SOILL')/

For example, the first set of numbers, 129, 240, also appears here:

data paramlist(704) /gribparam(129, 240, 0, 16, 197, 'RETOP')/

edwardhartnett commented 2 years ago

@BoiVuong-NOAA so then it's OK for duplicate GRIB1 numbers to exists.

Multiple parameters can be be indicated by the same GRIB1 numbers.

However, when looking up the parameter, only the first one will be found. So even if the user intends to mean:

data paramlist(1016) /gribparam(129, 240, 0, 16, 3, 'RETOP') The library will think he means:

data paramlist(704) /gribparam(129, 240, 0, 16, 197, 'RETOP')/ which has different GRIB2 numbers.

So a round trip translation from GRIB1 to GRIB2 back to GRIB1 would not work in this case.

BoiVuong-NOAA commented 2 years ago

The developer used "RETOP" in "user local define" for developing at the time this parameter e.g., RETOP was not approved by WMO. After that, We received an approval from WMO (may be couple months or a year)

edwardhartnett commented 2 years ago

Which one is which? THat is, which of the two RETOPs is the locally-defined one, and which is the WMO-defined one?

BoiVuong-NOAA commented 2 years ago

Yes, You are correct. The locally-defined one has a number larger than 192 (e.g, 193, 194,...) and the WMO-defined one has a number from 00 - 191.

edwardhartnett commented 2 years ago

OK, but since the list is searched from the beginning to end:

  subroutine param_g1_to_g2(g1val, g1ver, g2disc, g2cat, g2num)
    implicit none

    integer, intent(in) :: g1val, g1ver
    integer, intent(out) :: g2disc, g2cat, g2num
    integer :: n

    g2disc = 255
    g2cat = 255
    g2num = 255

    do n = 1, MAXPARAM
       if (paramlist(n)%grib1val .eq. g1val .AND. &
            paramlist(n)%g1tblver .eq. g1ver ) then
          g2disc = paramlist(n)%grib2dsc
          g2cat = paramlist(n)%grib2cat
          g2num = paramlist(n)%grib2num
          return
       endif
    enddo

So if a user calls param_g1_to_g2() on the GRIB1 numbers 120, 240, they will get the first RETOP, which is:

data paramlist(704) /gribparam(129, 240, 0, 16, 197, 'RETOP')

In other words, they will get the locally defined GRIB2 parameter. Don't we want them to instead get the WMO-defined one?

BoiVuong-NOAA commented 2 years ago

The purpose of params.f file is for conversion from grib2 to grib1. The search routine will not get the second one once it found the first one. They will get locally defined GRIB2 parameter.

edwardhartnett commented 2 years ago

OK so we agree that the search routine will find the locally-defined GRIB2 parameter.

But is that what should happen? Don't we want the routine to instead find the WMO parameter, which would be more generally understood, instead of the locally-defined parameter, which will only be understood within NOAA?

BoiVuong-NOAA commented 2 years ago

The WMO-defined one is only be understood within NOAA in GRIB2. If some one wants to convert from grib2 to grib1, then the grib1 still give them a correct one in grib1.

edwardhartnett commented 2 years ago

Why would the WMO one only be understood within NOAA?

The WMO one will also be understood by ECMWF right?

BoiVuong-NOAA commented 2 years ago

Why would the WMO one only be understood within NOAA ? Because our developer uses in our NCEP WMO for all developing works in AWIPS products. Once the AWIPS products go in operational, we will a hard time to change it when we received an official WMO. You are right. The WMO one will also be understood by ECMWF right. I do not ECMWF has a locally define or not.

edwardhartnett commented 2 years ago

OK, so even if we wanted to change from the locally-defined one to the WMO one, that would not work because of AWIPS?

Who develops the AWIPS products at NOAA?

BoiVuong-NOAA commented 2 years ago

I do not know all of them. I know GFS who is Andrew POC after my retirement date. But, you can send a message out or the best way is to contact NCO to let them know you are going to make some changes in parameters in GRIB2 filess. Note: AWIPS products require a notice or TIN for 30-days or 90-days for any changes in AWIPS.

edwardhartnett commented 2 years ago

OK thanks. I will try to avoid making work for AWIPS for sure!

This notice is only required for GRIB1 changes parameter changes right?

BoiVuong-NOAA commented 2 years ago

The notice requires for any changes in production ( not only in grib1 or grib2). For AWIPS, the notice needs for 30-days or 90-days for any changes in AWIPS products. Note: The AWIPS products now are only in GRIB2. If there some AWIPS products which need to use in grib1. They need to use grib_util conversion program (cnvgrib).

edwardhartnett commented 5 months ago

@Hang-Lei-NOAA and @AlexanderRichert-NOAA what strikes me is this:

But, you can send a message out or the best way is to contact NCO to let them know you are going to make some changes in parameters in GRIB2 filess. Note: AWIPS products require a notice or TIN for 30-days or 90-days for any changes in AWIPS.

Do we need to be sending notices of this kind for our changes to the library? Of any NCEPLIBS libs?

AlexanderRichert-NOAA commented 5 months ago

My understanding is that when there are product changes in ops models, those have to be announced X amount of time in advance. So I don't think we have a direct obligation here, it's up to model developers+SPAs to identify/announce changes (which we can indirectly facilitate by letting developers know about changes that will affect file content).

edwardhartnett commented 5 months ago

What is an "SPA"?

We are making changes in the software stack, and testing is certainly appropriate.

What I would really like to do is contact these developers, find out exactly what testing they do, and see if we can set that up as automated testing for our products. In other words, run their tests for them in the CI system. Then we can be sure not to break it.

The follow-on step would be to work with the AWIPS people to improve testing even further.

Who are the developers?

Hang-Lei-NOAA commented 5 months ago

These are things that we don't need to care much about. Both our libs, 3rd party libs, and models transferred to NCO requires the release notes. The release notes will provide the detailed content of what have been changed and its impacts. I was a global workflow code manger. Usually, for the changes in model results are major issue in the release notes of models. (GFS). But for our libraries, we only need to notice the model team and assist them to adjust their models to correctly use it. The changes in results will be reported by them.

That is also the reason why the operational machines need to keep different version of libraries. Each release of models may use their appropriate libraries. Not always update the use of new libraries.

On Mon, Jun 10, 2024 at 4:21 AM Edward Hartnett @.***> wrote:

What is an "SPA"?

We are making changes in the software stack, and testing is certainly appropriate.

What I would really like to do is contact these developers, find out exactly what testing they do, and see if we can set that up as automated testing for our products. In other words, run their tests for them in the CI system. Then we can be sure not to break it.

The follow-on step would be to work with the AWIPS people to improve testing even further.

Who are the developers?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-g2/issues/311#issuecomment-2157679489, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFHFI6FCJGFY5WND2L3ZGVOYZAVCNFSM6AAAAABJA6F3PCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJXGY3TSNBYHE . You are receiving this because you were mentioned.Message ID: @.***>