New section 4 templates to encode 2D wave spectra and wave parameters selected by wave period

sebvi commented 2 years ago

Initial request

ECMWF is engaged in migrating its operations to GRIB2 with the aim to complete the work in the following years. To support this activity, we are requesting new section 4 templates in the context of wave modelling to encode some products, namely the full 2D wave spectra (wave direction and frequency) and products based on a subset of waves discriminated by wave periods and wave period ranges.

Although the parameters for wave spectra exists in Code Table 4.2, discipline 10 - oceanography, category 0 - waves, there are no templates in section 4 in which one can specify the partitioning of the spectra in terms of frequency and direction.

At ECMWF, the current partitioning of the spectra consists of 36 directions (every 10 degrees, 0 to 350) and 36 frequencies.

The proposed templates 4.99 and 4.100 enable the encoding of wave spectra for deterministic and ensemble forecasting respectively and allow to specify the full list of frequencies and directions of the partitioning (in a similar way the hybrid levels coefficients can be specified at the end of section 4). This is handy when frequencies and/or directions can not be derived from a mathematical formula.

The proposed templates 4.101 and 4.102 enable the encoding of wave spectra for deterministic and ensemble forecasting respectively and allow to specify the mathematical formulae needed to derive the frequencies and directions programatically. At ECMWF, the directions can be derived using an arithmetic formula while the frequencies can be derived using a geometric formula. The formula used can be specified in a new Code table 4.251 and the parameters of the formula can be specified subsequently. This mechanism is similar to the mechanism used to specify aerosol particle size distributions.

Finally, the proposed templates 4.101 and 4.1102 enable the encoding of parameters based on a subset of the wave periods. For instance, at ECMWF, we produce the products several "significant wave height" with period ranges: 10 to 12 seconds, 12 to 14 seconds, 14 to 17 seconds ,etc. We also produce "mean wave period" with various threshold, for instance "period of at least 10 seconds", of at least 12 seconds, etc. The proposed templates allow the encoding of these ranges and thresholds.

Amendment details

ADD the following templates entries in code table 4.0 and update range of unused entries:

Code	Description
99	Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions
100	Individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions
101	Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with frequencies and directions defined by formulae
102	Individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with frequencies and directions defined by formulae
103	Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for waves selected by period range
104	Individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for waves selected by period range

ADD New Code Table 4.251 – Wave direction and frequency formulae

Code	Description
0	Undefined sequence
1	Geometric sequence (see Note 1)
2	Arithmetic sequence (see Note 2)
3-191	Reserved
192-254	Reserved for local use
255	Missing

Note 1: Geometric sequence: x_n = x_0 * r^(n-1) with "x_0" first parameter and "r" second parameter Note 2: Arithmetic sequence: a_n = a_1 + (n-1) d with "a_1" first parameter and "d" second parameter

ADD Template 4.99 - Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions

Octet	Description
10	Parameter Category (see code table 4.1)
11	Parameter Number (see code table 4.2)
12-13	Wave Direction Number
14-15	Number of Wave Directions – ND
16-17	Wave Frequency Number
18-19	Number of Wave Frequencies – NF
20	Type of Generating Process (see code table 4.3)
21	Background Process (defined by originating centre)
22	Generating Process Identifier (defined by originating centre)
23-24	Hours After Data Cut-off (see Note)
25	Minutes After Data Cut-off
26	Indicator of Unit of Time Range (see code table 4.4)
27-30	Forecast Time
31	Scale Factor of Wave Directions
	The following entry of 4 octets is repeated ND times. (ND is defined in octets 14-15)
32 + (ND-1)4 –35 + (ND-1)4	Scaled Values of Wave Directions
36 + (ND-1)*4	Scale Factor of Wave Frequencies
	The following entry of 4 octets is repeated NF times. (NF is defined in octets 18-19)
37 + (ND-1)4 + (NF-1)4 – 40 +(ND-1)4 + (NF-1)4	Scaled Values of Wave Frequencies

ADD Template 4.100 Individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions

Octet	Description
10	Parameter Category (see code table 4.1)
11	Parameter Number (see code table 4.2)
12-13	Wave Direction Number
14-15	Number of Wave Directions – ND
16-17	Wave Frequency Number
18-19	Number of Wave Frequencies – NF
20	Type Of Generating Process (see code table 4.3)
21	Background Process (defined by originating centre)
22	Generating Process Identifier (defined by originating centre)
23-24	Hours After Data Cut-off (see Note)
25	Minutes After Data Cut-off
26	Indicator of Unit of Time Range (see code table 4.4)
27-30	Forecast Time
31	Type of Ensemble Forecast (see code table 4.6)
32	Perturbation Number
33	Number of Forecasts in ensemble
34	Scale Factor of Wave Directions
	The following entry of 4 octets is repeated ND times. (ND is defined in octets 14-15)
35 + (ND-1)4 –38 + (ND-1)4	Scaled Values of Wave Directions
39 + (ND-1)*4	Scale Factor of Wave Frequencies
	The following entry of 4 octets is repeated NF times. (NF is defined in octets 18-19)
40 + (ND-1)4 + (NF-1)4 – 43 +(ND-1)4 + (NF-1)4	Scaled Values of Wave Frequencies

ADD Template 4.101 Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with frequencies and directions defined by formulae

Octet	Description
10	Parameter Category (see code table 4.1)
11	Parameter Number (see code table 4.2)
12-13	Wave Direction Number
14-15	Number of Wave Directions – ND
16-17	Wave Frequency Number
18-19	Number of Wave Frequencies – NF
20	Type of Generating Process (see code table 4.3)
21	Background Process
22	Generating Process Identifier
23-24	Hours After Data Cut-off
25	Minutes After Data Cut-off
26	Indicator of Unit of Time Range (see code table 4.4)
27-30	Forecast Time
31	Type of Wave Direction Sequence (see code table 4.251)
32	Number of Wave Direction Sequence Parameters – NDSP
	The next two entries are repeated NDSP times to encode the scale factors and scaled values of the sequence parameters.
33 + (NDSP-1)*5	Scale Factor of Wave Direction Sequence Parameter
34+(NDSP-1)5 - 37+(NDSP-1)5	Scaled Value of Wave Direction Sequence Parameter
33+NDSP*5	Type of Wave Frequency Sequence (see code table 4.251)
34+NDSP*5	Number of Wave Frequency Sequence Parameters – NFSP
	The next two entries are repeated NFSP times to encode the scale factors and scaled values of the sequence parameters.
35+NDSP5+(NFSP-1)5	Scale Factor of Wave Frequency Sequence Parameter
36+NDSP5+(NFSP-1)5 -39+NDSP5+(NFSP-1)5	Scaled Value of Wave Frequency Sequence Parameter

ADD Template 4.102 Individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with frequencies and directions defined by formulae

Octet	Description
10	Parameter Category (see code table 4.1)
11	Parameter Number (see code table 4.2)
12-13	Wave Direction Number
14-15	Number of Wave Directions – ND
16-17	Wave Frequency Number
18-19	Number of Wave Frequencies – NF
20	Type of Generating Process (see code table 4.3)
21	Background Process
22	Generating Process Identifier
23-24	Hours After Data Cut-off
25	Minutes After Data Cut-off
26	Indicator of Unit of Time Range (see code table 4.4)
27-30	Forecast Time
31	Type of Ensemble Forecast (see code table 4.6)
32	Perturbation Number
33	Number of Forecasts in Ensemble
34	Type of Wave Direction Sequence (see code table 4.251)
35	Number of Wave Direction Sequence Parameters – NDSP (can be set to zero if no parameters are required)
	The next two entries are repeated NDSP times to encode the scale factors and scaled values of the sequence parameters.
36 + (NDSP-1)*5	Scale Factor of Wave Direction Sequence Parameter
37+(NDSP-1)5 - 40+(NDSP-1)5	Scaled Value of Wave Direction Sequence Parameter
36+NDSP*5	Type of Wave Frequency Sequence (see code table 4.251)
37+NDSP*5	Number of Wave Frequency Sequence Parameters – NFSP (can be set to zero if no parameters are required)
	The next two entries are repeated NFSP times to encode the scale factors and scaled values of the sequence parameters.
38+NDSP5+(NFSP-1)5	Scale Factor of Wave Frequency Sequence Parameter
39+NDSP5+(NFSP-1)5 -42+NDSP5+(NFSP-1)5	Scaled Value of Wave Frequency Sequence Parameter

ADD Template 4.103 Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for waves selected by period range

Octet	Description
10	Parameter Category (see code table 4.1)
11	Parameter Number (see code table 4.2)
12	Type of Wave Period Interval (see code table 4.91)
13	Scale Factor of Lower Wave Period Limit
14 -17	Scaled Value of Lower Wave Period Limit
18	Scale Factor of Upper Wave Period Limit
19-22	Scaled Value of Upper Wave Period Limit
23	Type of Generating Process (see code table 4.3)
24	Background Process
25	Generating Process Identifier
26-27	Hours After Data Cut-off
28	Minutes After Data Cut-off
29	Indicator of Unit of Time Range (see code table 4.4)
30-33	Forecast Time
34	Type of First Fixed Surface (see code table 4.5)
35	Scale Factor of First Fixed Surface
36-39	Scaled Value of First Fixed Surface
40	Type of Second Fixed Surface (see code table 4.5)
41	Scale Factor of Second Fixed Surface
42-45	Scaled Value of Second Fixed Surface

ADD Template 4.104 Individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for waves selected by period range

Octet	Description
10	Parameter Category (see code table 4.1)
11	Parameter Number (see code table 4.2)
12	Type of Wave Period Interval (see code table 4.91)
13	Scale Factor of Lower Wave Period Limit
14-17	Scaled Value of Lower Wave Period Limit
18	Scale Factor of Upper Wave Period Limit
19-22	Scaled Value of Upper Wave Period Limit
23	Type of Generating Process (see code table 4.3)
24	Background Process
25	Generating Process Identifier
26-27	Hours After Data Cut-off
28	Minutes After Data Cut-off
29	Indicator of Unit of Time Range (see code table 4.4)
30-33	Forecast Time
34	Type of First Fixed Surface (see code table 4.5)
35	Scale Factor of First Fixed Surface
36-39	Scaled Value of First Fixed Surface
40	Type of Second Fixed Surface (see code table 4.5)
41	Scale Factor of Second Fixed Surface
42-45	Scaled Value of Second Fixed Surface
46	Type of Ensemble Forecast (see code table 4.6)
47	Perturbation Number
48	Number of Forecasts in Ensemble

Comments

No response

Requestor(s)

Jean Bidlot, Robert Osinski and Sebastien Villaume (all ECMWF)

Stakeholder(s)

Enter list of stakeholder(s).

Publication(s)

Manual on Codes (WMO-No. 306), Volume I.2, GRIB Code Table 4.0 (new entries) Manual on Codes (WMO-No. 306), Volume I.2, GRIB Code Table 4.251 (new table) Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.99 (new template) Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.100 (new template) Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.101 (new template) Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.102 (new template) Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.103 (new template) Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.104 (new template)

Expected impact of change

No response

Collaborators

The proposal was drafted together with the wave model Team at ECMWF and tested in ECMWF's GRIB decoder ecCodes

References

No response

Validation

No response

amilan17 commented 2 years ago

https://github.com/wmo-im/CCT/wiki/Teleconference-4.10.2022 meeting notes:

Sebastien presented this proposal; when ready Sebastien can provide examples and a pre-release of ecCodes for validation. @SibylleK will review proposal

sebvi commented 1 year ago

Here is a link to the experimental version of ecCodes able to encode/decode the new templates: eccodes

Let me know if you do not have access.

EDIT for correct sample: waves.zip

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference-1-and-2.11.2022 notes:

@SibylleK: received some comments from colleagues and will send to @sebvi

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference-1-and-2.11.2022 notes:

@amilan17 will work on

sergioh-pessoal commented 1 year ago

Dear @sebvi I downloaded the samples and source code from Eccodes. This works fine. As I am not an expert on waves, I asked a colleague, Rosio Maita, to take a look at the data, and another colleague, Fernando Ii, to help plotting with Metview. At the moment, we only draw a map with a string of '1'. We are trying to understand how to plot this data correctly, to verify the values.

shahramn commented 1 year ago

Dear @sebvi I downloaded the samples and source code from Eccodes. This works fine. As I am not an expert on waves, I asked a colleague, Rosio Maita, to take a look at the data, and another colleague, Fernando Ii, to help plotting with Metview. At the moment, we only draw a map with a string of '1'. We are trying to understand how to plot this data correctly, to verify the values.

Dear @sergioh-pessoal The GRIB files do not have actual data ( in the Data Section ). They are "constant fields" (all field values set to 1) This was done to reduce their size.

sergioh-pessoal commented 1 year ago

Dear @shahramn So the decoding is correct. thanks for the clarification

sebvi commented 1 year ago

We have receive several questions and comments from @SibylleK . We have worked on these and will provide a response as soon as possible (I need to reformat our answers/clarifications). It should be done this week.

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference-21.11.2022 notes: @sebvi will update the issue with questions and responses received offline. Will probably not change the proposal, but it's still in progress. @SibylleK said it would be useful to compare to the GRIB1 data file.

sebvi commented 1 year ago

Here are the questions received from @SibylleK and DWD.

The new Product Definition Templates (PDTs) 4.99 to 4.102 do not contain “fixed surface”-information: Maybe at least the titles of the templates should state that they have to do with water surface waves.

These templates are intended to be used exclusively for encoding 2D wave spectra. We think it is not needed to add fixed surface-information to these templates so we agree that the title of the templates could contain water surface waves.

Why do the new PDTs 4.103 and 4.104 contain “fixed surface”-information?

These templates are intended to encode parameters like the significant wave height which is related to the water surface. For this reason, the fixed surface-information was added.

-The (non-expert) users may not know if the wave frequency means “1/T” or “2p/T”, where “T” denotes the wave period. This should be clarified in a Note, but see also issue 6 below.

f=1/T is used and we are not aware that someone in the wave model community uses 2p/T. We will add a note.

-The (non-expert) users may not know on which wave dispersion relation the data are based, so they may be unable to assign a wavelength to the given wave frequencies (or wave periods). Maybe there should be at least an optional list of the wavelengths that correspond to the wave frequencies (or wave periods).

The dispersion relation is ω2 = g k tanh(k D). It includes the water depth which makes the relation frequency / wavelength, water depth dependent. If a user would need to determine the associated wave length, the model water depth is needed. An optional list of wavelength corresponding to the wave frequencies/periods would therefore not be valid over the entire model domain except the model would only consider deep water areas.

In order to be more specific and more flexible PDTs 4.99 to 4.102 should code wave direction and wave frequency (wave period) intervals instead of single direction and frequency values. Single values could still be coded by using:
“CT4.91: Type of Interval = 7 = Between first and second. The range includes the first limit and the second limit” plus “First interval limit = Second interval limit”

These templates are intended to encode 2D wave spectra. These spectra are based on discrete frequencies and directions. We are not aware of any wave model which uses intervals / bins of wave frequencies / directions to describe the wave spectrum. Do you have a specific use case in mind, which would make it necessary to encode intervals?

Wouldn’t it be ‘more straightforward and accessible’, also in view of your proposal for PDTs 4.103 and 4.104, to code wave periods (in seconds) instead of wave frequencies? In combination with the previous issue, one could then do without the new PDTs 4.103 and 4.104 in the following way: “Number of wave direction intervals – ND = 1” “First direction interval limit = 0 degree true” “Second direction interval limit = 360 degree true”

Maybe we should have split this proposal into two independent proposals because the first 4 templates are totally independent of the last 2 templates and serves completely different purpose, they should not be consider together. The wave spectrum is well established and comes from the simple assumption that the wave fields (all waves together) can be represented as Fourier series, for which frequency is normally used. We prefer to have separate templates for the 2D spectra (4.99-4.102) and for parameters which are derived(4.103/104) like the significant wave height (swh) to distinguish between spectrum and derived parameters.
Templates 4.103/104 are intended to be used for parameters like the “Significant wave height of all waves with periods within the inclusive range from 10 to 12 seconds“,`

The description of this parameter in our parameter database describes this parameter like this:

„This parameter is four times the square root of the integral over all directions of the two-dimensional wave spectrum and all frequencies between 1/12 and 1/10 hertz (i.e. periods between 10 and 12 seconds)“.

-There should be a Note on the unit of the wave direction.

We will add a note on the unit.

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference-12-and-13.12.2022 notes:

double check after branch update

amilan17 commented 1 year ago

Reviewed consistency and only needed to make many words all lowercase.

A couple templates are missing a note OR the reference to the note should be removed. "23-24,Hours of observational data cut-off after reference time (see Note),,,,,

sebvi commented 1 year ago

Thanks @amilan17 I can have a look at this. I believe the note already exists and have an note_id, it is the generic note we have for these octets in all templates.

amilan17 commented 1 year ago

@sebvi is it note ID 33?

33	Hours greater than 65534 will be coded as 65534.

amilan17 commented 1 year ago

@sebvi In this https://github.com/wmo-im/GRIB2/issues/167#issuecomment-1332454032 you mentioned that you would add some notes. Is this done?

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference-10.01.2023 notes:

@SibylleK - will verify notes

@amilan17 and @marijanacrepulja will review consistency

SibylleK commented 1 year ago

Notes have been verified.

wmo-im / GRIB2