search

wmo-im / wis2pilot

https://wmo-im.github.io/wis2pilot
Apache License 2.0
1 stars 3 forks source link

WP8.1 - GTS -> WIS2 Gateway #10

Open efucile opened 2 years ago

efucile commented 2 years ago

26 June 2023

DECISION

Haddouch101 commented 1 year ago
Haddouch101 commented 1 year ago

ET-W2AT 15-05-2023:

golfvert commented 1 year ago

Let's review this in an upcoming W2AT meeting before closing it.

josusky commented 1 year ago

BTW, there is another proposal to solve the same problem: add property for TTAAii

KenRJTD commented 1 year ago

Additional (Recommended) data/products are circulated over the GTS. Should these restricted data be published to WIS2? or only Essential (Core) data be published to WIS2.0?

golfvert commented 1 year ago

Everything that is currently exchanged over the GTS should be made available on WIS2 during the transition by the gateway(s). The principle is not to "loose" any data during the transition. The question though is how to recognize a "recommended" data on the GTS ? We probably don't want that the gateway(s) to "make" them core at their level.

KenRJTD commented 1 year ago

Res.40 Additional data (pdf files) are available in the middle of WMO web page. https://community.wmo.int/en/activity-areas/operational-information-service/additional-data-and-products I know Res.40 has already been replaced with New WMO data policy. I haven't seen any list of Recommended data so far. I think we will recognize recommended data by WIS2 metadata in the near future. My question is, as of today, don't Gateways need to care for the Res.40 additional data?

antje-s commented 1 year ago

A first solution was included in GTStoWIS2 TableA https://github.com/wmo-im/GTStoWIS2/blob/main/GTStoWIS2/TableA.json but this is no longer included after the changeover to TTAAii. Perhaps we could differentiate in the pub script itself depending on the TTAAii. But for this a list of all TTAAii headers with recommended data would be useful...as a first solution, we could orientate on the TableA and collect headers including "recomended". It should be ok to try it out and later the list could be completed/corrected.

antje-s commented 1 year ago

We updated topics to cache/a/wis2/xxg/gts_to_wis2_dwd/data/core/TTAAii/CCCC and cache/a/wis2/xxg/gts_to_wis2_dwd/data/recommended/TTAAii/CCCC

Attached the initial list of recommended TTAAii (startswith-check), which should be corrected/adjusted

FA FC FR FT ISV ISW IUA IUO JSA JSP JSS JST JTS KFA KFP KFS KFT KSV KSW KUA KUO L M SA SP UA UB UD UR UT V Y Z

kaiwirt commented 1 year ago

After discussion with @golfvert and @6a6d74 we decided that the gateways should publish under the origin TH and the data should be republished by the caches under the cache TH (i.e. GC JMA republishes DWD gateway gts data and DWD GC republishes JMA gateway gts data).

kaiwirt commented 1 year ago

The data id for the WNM from the GTS -> WIS2 Gateways should be unique for each gateway. That is JMA and DWD will choose different data ids for example UUID reflecting the fact, that GTS bulletins with the same header might be different at the different locations.

Therefore there will be two different GTS->WIS2 datasets (one for each gateway). The end-users receiving the GTS data through WIS2 will need to be able to handle duplicates

golfvert commented 1 year ago

After discussion during ET-W2AT the proposed topic hierarchy: cache/a/wis2/xxg/gts_to_wis2_dwd/data/recommended/T1/T2/A1/A2/ii/CCCC cache/a/wis2/xxg/gts_to_wis2_dwd/data/core/T1/T2/A1/A2/ii/CCCC

kaiwirt commented 1 year ago

Regarding https://github.com/wmo-im/wis2-topic-hierarchy/issues/58 what is suggested to use for the Gateway:

others?

golfvert commented 1 year ago

We are abandoning the xxa-g stuff. So, not the first one. Then, recognizing the role of dwd (and jma) I'd go for de-dwd-gts_to_wis2 or even de-dwd-gts-to-wis2 I wonder if we should restric the allowed characters in centre-id to [a-z],[0-9],- And nothing else.