add property for TTAAii

[tomkralidis]

Consider adding TTAAii (abbreviated header line) to message properties, i.e.:

"properties": {
  ...
  "ttaaii": "FTAE31"
  ...
}

[kaiwirt]

-1 from me. I thought we wanted to get rid of the headers.

From my point of view we should assume, that users of WIS2 don't know and don't need to know anything about headers. I am fine with translating headers to meaningful topics (maybe in a separate tree), but i would not add headers to messages or topics.

My problem is, that i am afraid that if we do something like that someone is going to use that "feature" and eventually the headers will stay in the messages forever.

The only reason for this i can somehow see is for the conversion of data back from WIS2 to GTS.

[golfvert]

I must confess I put this on the table exactly for "the conversion of data back from WIS2 to GTS"... I know it looks bad :) We haven't talk about the gateways for a long time. Having something like that would facilitate de WIS2 - > GTS gateway.Is putting this somewhere is the topic better? Not sure. Can we avoid this kind of mapping altogether? I am all ears for a better idea.

[kaiwirt]

This is a difficult question. If we choose this way we ask each WIS2 node to add a property to its messages saying "if this data would be sent out to the GTS it would have this header"

If we don't find an alternative for the WIS2GTS Gateways i would use the message. But we should make clear, that this property is solely used for that one purpose and that this property will be removed with the shutdown of the GTS. I am pretty sure that otherwise someone finds this property useful and we end up with one of those long living intermediate solutions.

[antje-s]

Maybe we can avoid it, because I think for the (old) GTS data there was a filename convention and if the filenames are not changed, TTAAii should be included in filenames, or am I wrong?

[josusky]

The notification format allows foreign key/properties, so if the GTS gateway infrastructure needs to add TTAAii somewhere for its own use, it can do so without frightening everybody by making it officially acknowledged as part of the standard. Just saying. I am not against mentioning it in the notification standard as an optional thing with a clear explanation that it is for internal use by the GTS <-> WIS2 gateways infrastructure and other (normal) WIS2 nodes shall not use it.

[tomkralidis]

Good point. We already have a clause for additional properties in the specification, so in theory we can keep as is, and then for specific GTS transition documentation, specify as needed. Thoughts?

[amilan17]

ET-W2AT meeting discussed this on 15 May and decided that to include TTAAii as an optional field is the simplist approach for supporting GTS\<->WIS2.

[golfvert]

As mentioned by Anna, it is considered (hopefully rightly so...) that in order to be able to implement the WIS -> GTS (and only that direction), having the GTS "metadata" (TTAAii and probably CCCC) as part of the message was an acceptable solution.

It has yet to be endorsed, but the following was discussed :

"properties": {
  ...
  "gts": {
       "ttaaii": "FTAE31",
       "cccc": "whatever"
  }
  ...
}

This has to be confirmed by GTS experts.

[tomkralidis]

Given the model of WNM allows for additional properties/specification as required, I would include this guidance only in the Guide itself (perhaps under an isolated "transition from GTS" section?). Such a section can then be managed more easily vs. sprinkling GTS legacy across WNM (and beyond?!).

Thoughts @josusky @kaiwirt @McDonald-Ian @ktsuboi-jma @antje-s @davidpodeur ?

[josusky]

I have a question for @golfvert or whoever has a clear picture of how the gateways will operate: "Who will be responsible for adding these properties?". That is, will we have some GTS->WIS2 gateways that will add them and then WIS2->GTS will use it? Or will we have just WIS2->GTS gateways and then all the producers will be responsible for adding these GTS properties to their notifications about data that are supposed to get back to GTS?

[golfvert]

Not sure I have a clear picture :) Considering that it would be extremely difficult (impossible ?) for a third party to map something received on WIS2 to its previous TTAAii/CCCC counterpart, the only practical solution is for the Data Producer to add those GTS references. The idea would be therefore to add this information somewhere. The most logical somewhere being the notification message. As part of the transition strategy, we agreed to have those gateways. We have to find the practical way to make them work. Having those properties seems to be reasonable to do for the data producer and should make the life of the gateway operators simpler.

[tomkralidis]

TT-WISMD 2023-05-22: this needs to be clarified/discussed further. Adding GTS specifics into WNM poses a risk in not being able to decommission GTS proper over time.

[josusky]

@golfvert we understand your position, our concern here is more political than technical. If we require the data producers (possibly most of them) to include these parameters in their notifications, the description of it must appear in the WIS manual, in a prominent place, otherwise, some producers "forget" to set them and the WIS2-->GTS gateway(s) will not work as expected.

[golfvert]

Agreed. We also have considered enforcing that when people will declare their WIS2 Node to WMO. Something part of the checklist. "You want to stop sending data using your MSS, OK, but you have to put this in the message". I think we all agree that this is not a "perfect" solution. It seems, however, a doable and acceptable one.

[kaiwirt]

Is it possible to give what @golfvert said as an option to users?

"During the transition phase from GTS to WIS2 you can either continue to send out GTS data through your message switch OR you can switch off your message switch IF you add GTS data identification to your WIS2 messages"

I can see advantages for centers that just want to keep their message switch running during the transition and not worry about TTAAii in WIS2. Conversely if someone wants to shut down their message switch they can do so if they continue to provide the GTS metadata.

[tomkralidis]

+1 to add as an optional property in WNM. The only nuance I would put forth is that this would be explained in a section in the Guide on "Migrating from GTS", which articulates the rules around, say, properties.gts, and not in WNM proper.

[KenRJTD]

Comment from a GTS expert. CCCC is necessary for GTS routing. For example Offenbach(EDZW) and Rome(LIIB) use same TTAAii (ISMD01) for their BUFR/SYNOP. Otherwise, they have same header and we may have problems, for example overwriting.

[josusky]

@KenRJTD Data instances will have a unique data_id, that is an existing requirement. In the transition period, the data_id might be constructed from the GTS abbreviated heading line elements but that is not necessary (not guaranteed). So these proposed properties.gts.ttaaii and properties.gts.cccc are meant to be added by data producers/publishers to help the operation of the WIS2->GTS gateway(s) during the migration period.

[antje-s]

Should we close the issue as agreed on adding properties.gts.ttaaii and properties.gts.cccc for GTS data during migration period?

[josusky]

Hm, there seems to be a decision allegedly from June 2023 in https://github.com/wmo-im/wis2pilot/issues/10 @tomkralidis are you aware of it? Does it mean that DWD (@antje-s ?) and JMA will provide the functionality, by publishing GTS data in a dedicated topic "branch", and the other WIS2 centres do not need to do anything?

[golfvert]

We collectively agreed that having those gateways (here GTS to WIS2) would facilitate the transition. DWD and JMA have agreed to operate those gateways. So, as a center, when the WIS2 node is installed, and if I provide the additional properties in the WIS2 notification message I can stop the GTS transmission. We don't require more gateways (if we have more it is not an issue). As a Center it gives me a choice. To stop or not my GTS system. That is the current situation.

[josusky]

Oh, so I misunderstood it. So the properties.gts.ttaaii and properties.gts.cccc shall be used by WIS2 centres that decide to stop GTS transmission, fair enough.

[golfvert]

If I am pedantic, I would write that providing those additional properties.gts in the notification message is a MUST for WIS2 Centres considering stopping their GTS transmission. Otherwise, yes :)

[tomkralidis]

@josusky given the agreement at the June 2023 W2AT meeting (didn't attend), I suggest we close this issue and have GTS to WIS2 related documentation (including WNM updates) bound to a dedicated transition document, and not part of WNM per se.

[amilan17]

TT-WISMD 2023-09-12:

re-discussed and verified that it is out of scope for WNM, but it should be part of the transition guide and they will finalize the WIS2 architecture in November

[6a6d74]

RE-opening this issue because inclusion of the TTAAii CCCC is not just a transition issue. ICAO SWIM needs the TTAAii CCCC "for the foreseeable future". I think we need the gts property to be mentioned in the WMN standard as conditional. Unless there's a better place for this? (BTW- I'm thinking about this because I'm writing the WIS2-SWIM interoperability section of the WIS2 Guide)

@tomkralidis @golfvert @amilan17

[tomkralidis]

@6a6d74 thanks for the info. Given 7.1.16, I would think the WIS2-SWIM interoperability section of the WIS2 Guide would also cover this. Then, when the time comes to decommission properties.gts, it would be done only int the Guide, which is a leaner/cleaner effort than WNM (in the Manual)?

[6a6d74]

Hi Tom. Could put it in the Guide - but think that inclusion of TTAAii and CCCC in the notification message needs to be normative for aviation weather data.

Another option would be to put a normative statement in the short doc Secretariat are compiling (have compiled?) outlining ongoing management of GTS headers for aviation.

@efucile - your thoughts?

[tomkralidis]

Thanks @6a6d74. +1 for including in the management of GTS headers for aviation documentation.

[6a6d74]

TT-WIS2-SWIM just met to discuss proposals. Apparently, the TTAAii CCCC is embedded in the METAR/TAF product anyway. The TTAAii CCCC is used as the product identifier.

Newer products, like QVACI, use a GML identifier as the unique identifier for the product. Again, this is embedded in the (IWXXM) product.

Summary: aviation weather does not need the GTS properties. But we do still need to add the GTS properties info to the Transition Guide. See wis2-transition-guide/issue#20.

[tomkralidis]

Superseded by https://github.com/wmo-im/wis2-transition-guide/issues/20#issue-2086105010