Composite local time in GRIB message

sebvi commented 4 years ago

Branch https://github.com/wmo-im/GRIB2/tree/issue-06

~ In the context of Fire Forecasting, we are creating composite GRIB messages of several UTC time.

Typically, Fire forecasting (risk of fire) is only relevant during day time. Usually, when encoding a GRIB message, the time is UTC and fixed for the entire grid domain (which is ok most of the time). For Fire forecasting however, if we were using universal time, then the data value would be relevant only for part of the horizontal domain. For instance, 12Z will be relevant for Europe but will be useless for Australia.

One could save several GRIB messages for several UTC time so that all continents are covered at some point with relevant, meaningful values. But it is much more practical to create composite GRIB messages that are composed by stripes of local times so that the data represents for instance the values for "noon" everywhere.

Practically, depending on the resolution of the model, we create GRIB files with 4 stripes (6 hour zones), 8 stripes (3 hours zones), 12 stripes (2 hours zones) or 24 stripes (hourly zones). The stripes are perfect longitudinal time zones (not following countries like the time zones we usually know).

stripes_local_time

The bottom part shows an example of 8 stripes (3 hours timestep) for a forecast starting at 00Z. each stripes in longitudes 360/8 degrees wide, i.e. 45 degrees: 337.5 to 22.5 degrees is 00z+12 UTC, 22.5 to 67.5 degrees is 00Z + 09 UTC, .... 292.5 to 337.5 degrees is 00Z+15 UTC

My feeling is that I need a whole new set of templates in section 4 that do not represent "point in time" or "continuous or non continuous time interval" but a set of keys to represent "composite time".

Alternatively an entry could be added in code table 1.2 to specify that the time in section 2 is still the reference time of the forecast (that still needs to be recorded) but that the time specified in section 4 will be a composite. The problem then is that I don't know how to specify the number of "stripes" which should also be recorded.

If anyone have suggestions, comments, alternatives?

sebvi commented 4 years ago

Please @wmo-im/tdcf have a look at this proposal :)

sebvi commented 4 years ago

@wmo-im/tdcf I have updated the issue with a picture and some extra explanations. Feedback, ideas are most welcome because we need to find the proper way to describe the metadata.

ckpan-hko commented 4 years ago

It's a good idea from user's perspective. But for GRIB products carrying global data, I think one single time domain would be better.

efucile commented 4 years ago

If I understand correctly there is a connection between time and grid. We can see it in different ways, but the connection is there. For different time steps we need to define a different slice of a global grid, connected with a different slice of data in the data section. This is very complex in GRIB and would require a careful implementation in the decoders. Not impossible, only very complex.

sebvi commented 4 years ago

Yes it is very complex! This is why I wanted an open discussion first before digging into this!

I could try to propose new sets of templates but it will be too short time to do so and too short notice to reasonably discuss it tomorrow. With the new github workflow, it seems much easier to me to consider approving requests more frequently than twice a year.

ckpan-hko commented 4 years ago

If we see "time" as a parameter with "UTC" as its base unit, does it mean that the "time" in different strips carries different unit?

sebvi commented 4 years ago

UPDATE

We continued to work on this proposal and here is where we are:

purpose of the proposal: Generate a template which allows to store an instantaneous field at a given local time using an appropriate forecast window to collate different UTC times

The simpler way to achieve this for each grid point and for a given solar/local time, is to:

calculate what is the exact UTC time at this location that corresponds to the chosen solar time
select grid point data (eg. Temperature) from a set of forecast step that is closest in UTC time to the exact calculated UTC time Following these simple rules, stripes of data are created and collated together. The better the time resolution of the input forecast, the narrower the stripes become and the less approximate is the solution. Basically We obtain schematically what is in the figures of the original post of this proposal.

However, this simple method generates artifacts as shown below with temperature where the strips are still visible (over Africa for instance).

A way to mitigate this is by interpolating data values to the given local time from the two closest UTC time steps as shown by the example below where a zoom is performed on one of the discontinuities.

time_interpolation

The temperature field is interpolated in the plot on the right and is simply concatenated (stripes) in the plot on the left. If we look at two points, 1 and 2, very close to the discontinuity and plot the time serie of the 24 hours forecast, in the case of the concatenation, those points will be attributed temperature from two different time stamps. Instead in the case of the interpolation values from two nearest time stamps are interpolated leading to a more accurate result.

Proposal

1) add a new entry in Code Table 1.2 Significance of reference time: "4 - Local Time" with a note explaining that the grid points represents the value of the parameters for the local time encoded in the reference time of the message. "local Time" should be used together with specific products Definition Templates in section 4 designed to be used with it and specifying how the data points were obtained and from which forecast steps.

2) create a products definition template in section 4 that describes what are the forecasts steps used and what methods has been used to obtain the parameter value (nearest forecast step (stripes), time interpolation, etc.)

We are in the process of drafting the template.

sebvi commented 4 years ago

Proposal

add a new entry in Code table 1.2 Significance of reference time Code	Meaning
4	Local Time

add in Code Table 4.0 an entry for the new template Code	Meaning
88	Analysis or Forecast at a horizontal level or in a horizontal layer at a local Time

add a new Product definition template 4.88 - Analysis or Forecast at a horizontal level or in a horizontal layer at a local Time Octet No.	Contents
10	Parameter category (see Code table 4.1)
11	Parameter number (see Code table 4.2)
12	Type of generating process (see Code table 4.3)
13	Background generating process identifier (defined by originating centre)
14	Forecast generating process identifier (defined by originating centre)
15	Type of first fixed surface (see Code table 4.5)
16	Scale factor of first fixed surface
17–20	Scaled value of first fixed surface
21	Type of second fixed surface (see Code table 4.5)
22	Scale factor of second fixed surface
23–26	Scaled value of second fixed surface
27	Method used to calculate the field value at the local time specified in section 1 (see Code table 4.248)
28	n - number of forecasts used in the processing
n/a	29-46 Specification of the first forecast used in the processing (reference date, reference time and forecast time increments)
29-30	Year of the reference date for the first forecast used in the processing
31	Month of the reference date for the first forecast used in the processing
32	Day of the reference date for the first forecast used in the processing
33	Hour of the reference time for the first forecast used in the processing
34	Minute of the reference time for the first forecast used in the processing
35	Second of the reference time for the first forecast used in the processing
36	Indicator of units of forecast time (see Code table 4.4)
37-40	Forecast time for the first forecast used in the processing, in units defined by the previous octet
41	Number of time increments for the first forecast used in the processing
42	Indicator of unit of time for the time increment between the successive forecast times used in the processing (see Code table 4.4)
43-46	Time increment between successive forecast times, in units defined by the previous octet
n/a	47-nn These octets are included only if n>1 where nn=28+18*n
47-nn	(n-1) repetitions of sequence of octet 29-46, describing the next forecasts used in the processing (reference date, reference time and forecast time increments)

create a new Code Table 4.248 Method used to derive the data values for a given local time Code figure	Meaning
0	Nearest forecast time to local time
1	Interpolated to be valid at the specified local time
2-191	Reserved
192 - 254	Reserved for local use
255	Missing

SimonElliottEUM commented 3 years ago

Left hand column of Code table 4.248 above should be "Code figure" rather than "Octet". But otherwise this look plausible to me

shahramn commented 3 years ago

Octet 28 : "n - number of forecast used in the processing" should read "n - number of forecasts used in the processing"