How to filter by spatial/temporal granularity to avoid unneccessary load on subscriber?

[maaikelimper]

when writing demo-applications that subscribe to the messages published on the GB we noted cases where an unexpected large amount of messages caused issues for the subscriber: Examples:

• Sweden publishing per-minute data
• Germany published large number of stations (high spatial density)

It made me wonder if we intend to set any limits on observation-data published in the .../data/core/... topic to avoid a single WIS2-node flooding unsuspecting subscribers with data ? Or should we have separate topic or message-attribute to limit subscriber to GBON data ?

[kaiwirt]

Related to https://github.com/wmo-im/wis2-guide/issues/14

[6a6d74]

I wonder if there's merit in publishing data like this through multiple "datasets"?

For example. Dataset A strictly adheres to Core data (spatial/temporal resolution, parameters etc.). Dataset B is a superset of Dataset A that is more "dense". Both could be published through WIS2. The notification messages from the originating centres could even refer to the same data object when it's included in both A and B. (Note that Global Caches might end up keeping 2 or more copies because data objects from A and B would logically be different).

Maybe Dataset B is marked as Recommended Data? Maybe not.

[golfvert]

I think the topic hierarchy is fine. We shouldn't add any filtering at the MQTT level. Filtering can be implemented at the download level. If the message includes the geometry, it could be a way to filter before download. I'd therefore suggest to close this issue.