wmo-im / wis2-notification-message

WIS 2.0 MQP message to notify users of availability of new data
https://wmo-im.github.io/wis2-notification-message
2 stars 2 forks source link

Find the best time to set properties.metadata_id as required #119

Open antje-s opened 4 months ago

antje-s commented 4 months ago

In the long term, the metadata should be required so that no data is disseminated for which there is no metadata. The GB monitoring metric wmo_wis2_gb_messages_no_metadata_total could be used to determine a good point in time. (It should be ensured that no fake metadata is accepted but only metadata that exists in the GDC.)

Open questions: 1) How to deal with differences in GDCs? (e.g. if the answer is negative (not existing), at least one other GDC should be requested) 2) Which threshold value should be used? Neither the data flow should be stopped nor should the limit value not be reached in real life (cf. WIS1)

kaiwirt commented 3 months ago

In my opinion this should be required before the start of the operational phase. Otherwise, setting this property might break an (then) operational data exchange.

tomkralidis commented 1 week ago

TT-WISMD 2024-10-25:

amilan17 commented 1 week ago

A change to the specification that requires all notification messages to include a metadata_id and then enforcing this validation by not accepting invalid notification messages would probably be a breaking change. Therefore, this will probably need to be introduced at a session with a very clear strategy for implementation.

josusky commented 1 week ago

I recently went through a metadata creation exercise with a small WIS2 node and it was pretty straight forward. The smaller is the center, the less metadata records it needs. The larger data providers have (need) more metadata records but also have more resources. All in all, making metadata_id required would not be as painful as it might seem. But a gradual approach is OK, we can start kindly asking data providers to add the metadata_id and make it a strong requirement later on.