New section 4 templates and tables for verification scores

[sebvi]

Initial request

ECMWF is interested in storing verification scores in GRIB2 in a near future.

With the existing GRIB2 templates, it is not possible to properly encode verification scores. The following templates are a proposal of how verification scores could be encoded in GRIB2.

We are not aiming to get this validated for FT 2024-2, we would like to start the discussion and eventually get it as experimental so that we can refine the proposal.

Amendment details

ADD to code table 4.0 Product definition template number Code	Description
137	Verification scores for analysis or forecast at a horizontal level or in a horizontal layer at a point in time
138	Verification scores for average, accumulation, and/or extreme values or other statistically processed values at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval
139	Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time
140	Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval
141	Verification scores for derived forecast based on all ensemble members at a horizontal level or in a horizontal layer at a point in time
142	Verification scores for derived forecasts based on all ensemble members at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

ADD Template 4.137 Verification scores for analysis or forecast at a horizontal level or in a horizontal layer at a point in time Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Additional Specification of verified parameter (see code table 4.119)
13	1	Type of Generating Process (see code table 4.3)
14	1	Background Process
15	1	Generating Process Identifier
16-17	2	Hours After Data Cut-off
18	1	Minutes After Data Cut-off
19	1	Indicator of Unit of Time Range (see code table 4.4)
20-23	4	Forecast Time
24	1	Type of First Fixed Surface (see code table 4.5)
25	1	Scale Factor of First Fixed Surface
26-29	4	Scaled Value of First Fixed Surface
30	1	Type of Second Fixed Surface (see code table 4.5)
31	1	Scale Factor of Second Fixed Surface
32-35	4	Scaled Value of Second Fixed Surface
36-37	2	Verification Score (see code table 4.120)
38	1	Type of Reference Dataset for Verification (see code table 4.121)
39	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
40	1	Type of Threshold Operator for Verification Score (see code table 4.91)
41	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
42	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=1:NA – can be 0 times if no parameters are necessary (na=0)
43+ (na-1)*5	1	Scale factor of additional Argument for Verification
44+ (na-1)5 - 47+(na-1)5	4	Scaled value of additional Argument for Verification
43+ NA5 - 44+ NA5	2	Year of Start Of Verification Period
45+ NA*5	1	Month of Start of Verification Period
46+ NA*5	1	Day of Start of Verification Period
47+ NA*5	1	Hour of Start of Verification Period
48+ NA*5	1	Minute of Start of Verification Period
49+ NA*5	1	Second of Start of Verification Period
50+ NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
51+ NA5+ (nv-1)11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
52+ NA5(nv-1)11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
53+ NA5(nv-1)11 - 56+ NA5(nv-1)11	4	Length of Time Range for Verification Period
57+ NA5(nv-1)11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
58+ NA5(nv-1)11 - 61+ NA5(nv-1)11	4	Time Increment for Verification Period
62+ NA5+NV11 - 63+ NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.138 Verification scores for average, accumulation, and/or extreme values or other statistically processed values at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Additional Specification of verified parameter (see code table 4.119)
13	1	Type of Generating Process (see code table 4.3)
14	1	Background Process
15	1	Generating Process Identifier
16-17	2	Hours After Data Cut-off
18	1	Minutes After Data Cut-off
19	1	Indicator of Unit of Time Range (see code table 4.4)
20-23	4	Forecast Time
24	1	Type of First Fixed Surface (see code table 4.5)
25	1	Scale Factor of First Fixed Surface
26-29	4	Scaled Value of First Fixed Surface
30	1	Type of Second Fixed Surface (see code table 4.5)
31	1	Scale Factor of Second Fixed Surface
32-35	4	Scaled Value of Second Fixed Surface
36-37	2	Year of End of Overall Time Interval
38	1	Month of end of Overall Time Interval
39	1	Day of end Of Overall Time Interval
40	1	Hour of end of Overall Time Interval
41	1	Minute of end of Overall Time Interval
42	1	Second of end of Overall Time Interval
43	1	Number of Time Range – NR
44-47	4	Number of Missing in Statistical Process
		The next six entries are repeated NR times nr=1:NR
48 + (nr-1)*12	1	Type of Statistical Processing (see code table 4.10)
49 + (nr-1)*12	1	Type of Time Increment (see code table 4.11)
50 + (nr-1)*12	1	Indicator of Unit for Time Range  (see code table 4.4)
51 + (nr-1)12  -  54 + (nr-1)12	4	Length of Time Range
55 + (nr-1)*12	1	Indicator of Unit for Time Increment (see code table 4.4)
56 + (nr-1)12  -  59 + (nr-1)12	4	Time Increment
		End of sequence repeated NT times
60+ (NR-1)12 - 61+ (NR-1)12	2	Verification Score (see code table 4.120)
62+ (NR-1)*12	1	Type of Reference Dataset for Verification (see code table 4.121)
63+ (NR-1)*12	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
64+ (NR-1)*12	1	Type of Threshold Operator for Verification Score (see code table 4.91)
65+ (NR-1)*12	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
66+ (NR-1)*12	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=1:NA – can be 0 times if no parameters are necessary (na=0)
67+ (NR-1)12 + (na-1)5	1	Scale factor of additional Argument for Verification
68+ (NR-1)12 + (na-1)5 - 71+ (NR-1)12 +(na-1)5	4	Scaled value of additional Argument for Verification
67+ (NR-1)12 + NA5 - 68+ (NR-1)12 + NA5	2	Year of Start Of Verification Period
69+ (NR-1)12 + NA5	1	Month of Start of Verification Period
70+ (NR-1)12 + NA5	1	Day of Start of Verification Period
71+ (NR-1)12 + NA5	1	Hour of Start of Verification Period
72+ (NR-1)12 + NA5	1	Minute of Start of Verification Period
73+ (NR-1)12 + NA5	1	Second of Start of Verification Period
74+ NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
75+ (NR-1)12 1+ NA5+ (nv-1)*11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
76+ (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
77+ (NR-1)12 + NA5(nv-1)11 - 80+ (NR-1)12 + NA5(nv-1)11	4	Length of Time Range for Verification Period
81+ (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
82+ (NR-1)12 + NA5(nv-1)11 - 85+ (NR-1)12 + NA5(nv-1)11	4	Time Increment for Verification Period
86+ (NR-1)12 + NA5+NV11 - 87+ (NR-1)12 + NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.139 Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Additional Specification of verified parameter (see code table 4.119)
13	1	Type of Generating Process (see code table 4.3)
14	1	Background Process
15	1	Generating Process Identifier
16-17	2	Hours After Data Cut-off
18	1	Minutes After Data Cut-off
19	1	Indicator of Unit of Time Range (see code table 4.4)
20-23	4	Forecast Time
24	1	Type of First Fixed Surface (see code table 4.5)
25	1	Scale Factor of First Fixed Surface
26-29	4	Scaled Value of First Fixed Surface
30	1	Type of Second Fixed Surface (see code table 4.5)
31	1	Scale Factor of Second Fixed Surface
32-35	4	Scaled Value of Second Fixed Surface
36	1	Type of Ensemble Forecast (see code table 4.6)
37-40	4	Perturbation Number
41-44	4	Number of Forecasts in Ensemble
45-46	2	Verification Score (see code table 4.120)
47	1	Type of Reference Dataset for Verification (see code table 4.121)
48	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
49	1	Type of Threshold Operator for Verification Score (see code table 4.91)
50	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
51	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=1:NA – can be 0 times if no parameters are necessary (na=0)
52+ (na-1)*5	1	Scale factor of additional Argument for Verification
53+ (na-1)5 - 56+(na-1)5	4	Scaled value of additional Argument for Verification
52+ NA5 - 53+ NA5	2	Year of Start Of Verification Period
54+ NA*5	1	Month of Start of Verification Period
55+ NA*5	1	Day of Start of Verification Period
56+ NA*5	1	Hour of Start of Verification Period
57+ NA*5	1	Minute of Start of Verification Period
58+ NA*5	1	Second of Start of Verification Period
59+ NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
60+ NA5+ (nv-1)11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
61+ NA5(nv-1)11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
62+ NA5(nv-1)11 - 65+ NA5(nv-1)11	4	Length of Time Range for Verification Period
66+ NA5(nv-1)11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
67+ NA5(nv-1)11 - 70+ NA5(nv-1)11	4	Time Increment for Verification Period
71+ NA5+NV11 - 72+ NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.140 Verification scores for individual ensemble forecast, control and perturbed, at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Additional Specification of verified parameter (see code table 4.119)
13	1	Type of Generating Process (see code table 4.3)
14	1	Background Process
15	1	Generating Process Identifier
16-17	2	Hours After Data Cut-off
18	1	Minutes After Data Cut-off
19	1	Indicator of Unit of Time Range (see code table 4.4)
20-23	4	Forecast Time
24	1	Type of First Fixed Surface (see code table 4.5)
25	1	Scale Factor of First Fixed Surface
26-29	4	Scaled Value of First Fixed Surface
30	1	Type of Second Fixed Surface (see code table 4.5)
31	1	Scale Factor of Second Fixed Surface
32-35	4	Scaled Value of Second Fixed Surface
36	1	Type of Ensemble Forecast (see code table 4.6)
37-40	4	Perturbation Number
41-44	4	Number of Forecasts in Ensemble
45-46	2	Year of End of Overall Time Interval
47	1	Month of end of Overall Time Interval
48	1	Day of end Of Overall Time Interval
49	1	Hour of end of Overall Time Interval
50	1	Minute of end of Overall Time Interval
51	1	Second of end of Overall Time Interval
52	1	Number of Time Range – NR
53-56	4	Number of Missing in Statistical Process
		The next six entries are repeated NR times nr=1:NR
57 + (nr-1)*12	1	Type of Statistical Processing (see code table 4.10)
58 + (nr-1)*12	1	Type of Time Increment (see code table 4.11)
59 + (nr-1)*12	1	Indicator of Unit for Time Range  (see code table 4.4)
60 + (nr-1)12  - 63 + (nr-1)12	4	Length of Time Range
64+ (nr-1)*12	1	Indicator of Unit for Time Increment (see code table 4.4)
65 + (nr-1)12  -  68 + (nr-1)12	4	Time Increment
		End of sequence repeated NT times
69+ (NR-1)12 - 70+ (NR-1)12	2	Verification Score (see code table 4.120)
71+ (NR-1)*12	1	Type of Reference Dataset for Verification (see code table 4.121)
72+ (NR-1)*12	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
73+ (NR-1)*12	1	Type of Threshold Operator for Verification Score (see code table 4.91)
74+ (NR-1)*12	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
75+ (NR-1)*12	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=1:NA – can be 0 times if no parameters are necessary (na=0)
76+ (NR-1)12 + (na-1)5	1	Scale factor of additional Argument for Verification
77+ (NR-1)12 + (na-1)5 - 80+ (NR-1)12 +(na-1)5	4	Scaled value of additional Argument for Verification
76+ (NR-1)12 + NA5 - 77+ (NR-1)12 + NA5	2	Year of Start Of Verification Period
78+ (NR-1)12 + NA5	1	Month of Start of Verification Period
79+ (NR-1)12 + NA5	1	Day of Start of Verification Period
80+ (NR-1)12 + NA5	1	Hour of Start of Verification Period
81+ (NR-1)12 + NA5	1	Minute of Start of Verification Period
82+ (NR-1)12 + NA5	1	Second of Start of Verification Period
83+ NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
84+ (NR-1)12 1+ NA5+ (nv-1)*11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
85+ (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
86+ (NR-1)12 + NA5(nv-1)11 - 89+ (NR-1)12 + NA5(nv-1)11	4	Length of Time Range for Verification Period
90+ (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
91+ (NR-1)12 + NA5(nv-1)11 - 94+ (NR-1)12 + NA5(nv-1)11	4	Time Increment for Verification Period
95+ (NR-1)12 + NA5+NV11 - 96+ (NR-1)12 + NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.141 Verification scores for derived forecast based on all ensemble members at a horizontal level or in a horizontal layer at a point in time Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Additional Specification of verified parameter (see code table 4.119)
13	1	Type of Generating Process (see code table 4.3)
14	1	Background Process
15	1	Generating Process Identifier
16-17	2	Hours After Data Cut-off
18	1	Minutes After Data Cut-off
19	1	Indicator of Unit of Time Range (see code table 4.4)
20-23	4	Forecast Time
24	1	Type of First Fixed Surface (see code table 4.5)
25	1	Scale Factor of First Fixed Surface
26-29	4	Scaled Value of First Fixed Surface
30	1	Type of Second Fixed Surface (see code table 4.5)
31	1	Scale Factor of Second Fixed Surface
32-35	4	Scaled Value of Second Fixed Surface
36	1	Derived Forecast (see code table 4.7)
37-40	4	Number of Forecasts in Ensemble
41-42	2	Verification Score (see code table 4.120)
43	1	Type of Reference Dataset for Verification (see code table 4.121)
44	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
45	1	Type of Threshold Operator for Verification Score (see code table 4.91)
46	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
47	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=1:NA – can be 0 times if no parameters are necessary (na=0)
48+ (na-1)*5	1	Scale factor of additional Argument for Verification
49+ (na-1)5 - 52+(na-1)5	4	Scaled value of additional Argument for Verification
48+ NA5 - 49+ NA5	2	Year of Start Of Verification Period
50+ NA*5	1	Month of Start of Verification Period
51+ NA*5	1	Day of Start of Verification Period
52+ NA*5	1	Hour of Start of Verification Period
53+ NA*5	1	Minute of Start of Verification Period
54+ NA*5	1	Second of Start of Verification Period
55+ NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
56+ NA5+ (nv-1)11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
57+ NA5(nv-1)11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
58+ NA5(nv-1)11 - 61+ NA5(nv-1)11	4	Length of Time Range for Verification Period
62+ NA5(nv-1)11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
63+ NA5(nv-1)11 - 66+ NA5(nv-1)11	4	Time Increment for Verification Period
67+ NA5+NV11 - 68+ NA5+NV11	2	Number of Forecasts in Verification

ADD Template 4.142 Verification scores for derived forecasts based on all ensemble members at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval Octet	Number of bytes	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Additional Specification of verified parameter (see code table 4.119)
13	1	Type of Generating Process (see code table 4.3)
14	1	Background Process
15	1	Generating Process Identifier
16-17	2	Hours After Data Cut-off
18	1	Minutes After Data Cut-off
19	1	Indicator of Unit of Time Range (see code table 4.4)
20-23	4	Forecast Time
24	1	Type of First Fixed Surface (see code table 4.5)
25	1	Scale Factor of First Fixed Surface
26-29	4	Scaled Value of First Fixed Surface
30	1	Type of Second Fixed Surface (see code table 4.5)
31	1	Scale Factor of Second Fixed Surface
32-35	4	Scaled Value of Second Fixed Surface
36	1	Derived Forecast (see code table 4.7)
37-40	4	Number of Forecasts in Ensemble
41-42	2	Year of End of Overall Time Interval
43	1	Month of end of Overall Time Interval
44	1	Day of end Of Overall Time Interval
45	1	Hour of end of Overall Time Interval
46	1	Minute of end of Overall Time Interval
47	1	Second of end of Overall Time Interval
48	1	Number of Time Range – NR
49-52	4	Number of Missing in Statistical Process
		The next six entries are repeated NR times nr=1:NR
53 + (nr-1)*12	1	Type of Statistical Processing (see code table 4.10)
54 + (nr-1)*12	1	Type of Time Increment (see code table 4.11)
55 + (nr-1)*12	1	Indicator of Unit for Time Range  (see code table 4.4)
56 + (nr-1)12  -  59 + (nr-1)12	4	Length of Time Range
60 + (nr-1)*12	1	Indicator of Unit for Time Increment (see code table 4.4)
61 + (nr-1)12  -  64 + (nr-1)12	4	Time Increment
		End of sequence repeated NT times
65+ (NR-1)12 - 66+ (NR-1)12	2	Verification Score (see code table 4.120)
67+ (NR-1)*12	1	Type of Reference Dataset for Verification (see code table 4.121)
68+ (NR-1)*12	1	Type of Statistical Processing over Vertical for Verification (see code table 4.10)
69+ (NR-1)*12	1	Type of Threshold Operator for Verification Score (see code table 4.91)
70+ (NR-1)*12	1	Kind of Additional Arguments for Verification Score (see code table 4.121)
71+ (NR-1)*12	1	Number of additional arguments for verification - NA
		The next two entries are repeated NA times na=1:NA – can be 0 times if no parameters are necessary (na=0)
72+ (NR-1)12 + (na-1)5	1	Scale factor of additional Argument for Verification
73+ (NR-1)12 + (na-1)5 - 76+ (NR-1)12 +(na-1)5	4	Scaled value of additional Argument for Verification
72+ (NR-1)12 + NA5 - 73+ (NR-1)12 + NA5	2	Year of Start Of Verification Period
74+ (NR-1)12 + NA5	1	Month of Start of Verification Period
75+ (NR-1)12 + NA5	1	Day of Start of Verification Period
76+ (NR-1)12 + NA5	1	Hour of Start of Verification Period
77+ (NR-1)12 + NA5	1	Minute of Start of Verification Period
78+ (NR-1)12 + NA5	1	Second of Start of Verification Period
79+ NA*5	1	Number of Verification Period Time Ranges - NV
		The next three entries are repeated NR times nv=1:NV
80+ (NR-1)12 1+ NA5+ (nv-1)*11	1	Type of Statistical Processing for Time Range for Verification Period (see code table 4.10)
81+ (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Range for Verification Period (see code table 4.4)
82+ (NR-1)12 + NA5(nv-1)11 - 85+ (NR-1)12 + NA5(nv-1)11	4	Length of Time Range for Verification Period
86+ (NR-1)12 + NA5(nv-1)*11	1	Indicator of Unit for Time Increment for Verification Period (see code table 4.4)
87+ (NR-1)12 + NA5(nv-1)11 - 90+ (NR-1)12 + NA5(nv-1)11	4	Time Increment for Verification Period
91+ (NR-1)12 + NA5+NV11 - 92+ (NR-1)12 + NA5+NV11	2	Number of Forecasts in Verification

ADD NEW code table 4.119 Additional specification of verified parameter Code	Description
0	Anomaly
1	Bias corrected before verification
2	Statistically post-processed before verification
3-254	Reserved
255	Missing

ADD NEW code table 4.120 Verification scores Code	Description
0	Mean error
1	Root mean squared error
2	Continuous ranked probability score (CRPS)
3	Fair continuous ranked probability score
4	Continuous logarithmic ignorance score
5	Fair continuous logarithmic ignorance score
6	Ranked probability score
7-99	Reserved for future use
100	Contingency Table (deterministic) - hits
101	Contingency Table (deterministic) - false alarms
102	Contingency Table (deterministic) - misses
103	Contingency Table (deterministic) - correct negatives
104	Contingency Table (probabilistic) - Forecast-by-n-members and observed
105	Contingency Table (probabilistic) - Forecast-by-n-members and not observed
106- 62999	Reserved
63000-65534	For experimental use at local level
65535	Missing

Note: The entries 7,8,9,10,11 and 12 need a threshold. The number of additional keys is equal to the number of arguments specified for the code number set regarding table 4.122 in the key ‘Kind of additional arguments for verification score’. For entries 11 and 12 and ensemble template must be used to define the total number of members and n.

ADD NEW code table 4.121 Type of reference dataset for verification Code	Description
0	Analysis interpolated on forecast grid
1	Analysis with forecast interpolated on analysis grid
2	Analysis interpolated on a common grid with the forecast but different to original grid
3	Gridded observation on forecast grid
4	Gridded observation with forecast interpolated on observation grid
5-254	Reserved
255	Missing

ADD NEW code table 4.122 Kind of additional arguments for verification score Code	Description
0	Fixed threshold value
1	Threshold specified by a quantile of the climatology
2	Threshold specified multiple of standard deviations of the climatology
3-254	Reserved
255	Missing

Note: For #1: NA =1 or 2 with one scale factor / scaled value for a fixed threshold value if ‘type of limit for verification score’ includes only one value otherwise 2 for lower / upper limit

For #2: NA=1 or 2 1st tuple of scale factor / scaled value defined the total number of quantiles / 2nd tuple the quantile value if ‘type of limit for verification score’ includes only one value otherwise 4 for lower quantile / upper quantile

For #3: NA=1or 2 with scale factor / scaled value specifying the factor of standard deviation of the climatology if ‘type of limit for verification score’ includes only one value otherwise 2 for lower / upper limit

Comments

No response

Requestor(s)

Sebastien Villaume (ECMWF) Robert Osinski (ECMWF)

Stakeholder(s)

ECMWF

Publication(s)

Manual on Code: new templates for section 4 Manual on Code: new tables and new entries in code Table 4.2

Expected impact of change

None

Collaborators

No response

References

No response

Validation

No response

[amilan17]

https://github.com/wmo-im/CCT/wiki/Teleconference.16.May.2024 notes: Sebastien introduced the proposal; Sibylle commented that there should always be two bytes;

[amilan17]

https://github.com/wmo-im/CCT/wiki/Teleconference.16.May.2024 notes: available for discussion probably for next fast-track;

[sebvi]

we have an ecCodes branch if someone is interested to test our proposed implementation and give feedback on this: feature/ft2024-2_255

[sebvi]

Hi @wmo-im/tt-tdcf ,

we haven't received comments or suggestions in relation to our prosposal for encoding verification scores. We are planning to go ahead with this during this fast track.

[edouardsandrine]

Really sorry for the delay.

I have a comment from one of our CCMEP experts:

In the table 4.120, it seems to be taken for granted that the contingency tables are 2x2 (hits, false alarms, misses and correct negatives). This seems restrictive to them. Shouldn't we consider swapping NxN tables whose 2x2 case is easy to obtain if need be by adding rows and columns so as to merge categories?

Thanks.

[sebvi]

Dear @edouardsandrine ,

thank you for your comments. I passed it on to our verification experts. Do you have examples of contingency tables of higher dimensions than 2x2? The new Code Table 4.120 would be able to cater for several contigency tables if it was needed.

[edouardsandrine]

I will try to find such examples and come back to you. Thanks for your comment!

[edouardsandrine]

Please, find below some examples from a reference for the protocol for exchanging performance metrics between centers that provides examples of 3x3 and 4x4 contingency tables:

https://confluence.ecmwf.int/display/WLD/Data+format+of+reports+of+station-based+scores

[amilan17]

https://github.com/wmo-im/tt-tdcf/wiki/2024.11.12.tt.tdcf notes: Sebastien believes that the proposal is ready for this FT; Jeff shared it with NCEP folks with no response so far; ECMWF uses these but currently in a different format

• [ ] @sebvi provide samples with eccodes branch
• [ ] validate