chanskw
commented
6 years ago @ddebrunner For data publish, I added a strategy for versioning of the data scheme in the health toolkit. Here's what I suggested there:
Publishing Data
Services employ a publish-subscribe model to receive or send data to another service. When publishing data, topic definition should follow these guidelines:
- The topic be defined using the MQTT convention as noted by the Publish/Subscribe operator in the streamsx.topology toolkit. (e.g. /a/b/c/d)
- The topic should begin with the name of the service, delimited by "/"
- Next the topic should describe the data being published
- The last segment of the topic should be a version number. For example: /com/ibm/streamsx/health/example/java/service/observations/v1
- All services publish data in JSON format, to maximize service interoperability with different languages. (This is optional)
- If the data schema needs to change as the service evolves, follow these guidelines:
- Adding a new attribute to the schema is not a breaking change. Update the minor number of the topic version as follows to indicate a change in the schema: e.g. a/b/c/observations/v1 -> /a/b/c/observations/v1/1
- Renaming or removing of an attribute breaks compatibility. If this has to be done, update the major number of the topic version as follows to indicate a breaking change: e.g. /a/b/c/observations/v2
- Renaming or removing attributes is discouraged as it breaks application compatibility. It is recommended that existing data stream is maintained. New data stream can be added to a microservice to maintain compatibility.
Subscribing Data
To subscribe data from an upstream application, a service should employ the following guidelines:
- If a service can only handle one version of the data being published, match the name of the topic exactly. e.g. /com/ibm/streamsx/health/example/java/service/observations/v1
- If a service can handle non-breaking changes in the data schema from an upstream application. A service should subscribe to data using the # wildcard. e.g. /com/ibm/streamsx/health/example/java/service/observations/v1/# - In this case, as long as the upstream application does not bump the major version number, the service can continue to receive data.
I think this should be part of the base guideline. I'd like to get you feedback on this.
I think the data topic and schema should be considered part of the service's API. Therefore we need a way to figure out how to evolve it.
ddebrunner
Added an initial implementation guidelines that I created/proposed a while ago.