Question: Forward compatibility between encoder code and schema version

[estaban]

I am using SBE to pass data between 2 services: service A encodes the message and service B decodes it. I have 3 code packages:

• Package 1 (used by service A and B): contains the codec code generated by the SBE tool
• Package 2 (used by service A): the client code, that encodes the message
• Package 3 (used by service B): the server code, that decodes the message

At the moment, I have version 0 of the message has no body. I am adding a new variable length field in version 1 of the message.

Let's assume I want to update service B to use the new message version. I update the decoding code in package 3 to deserialize the new field and push a change in both packages 1 and 3.

While I pushed the change in package 1, but have not yet update the encoder code in package 2, then package 2 will use message version 1, but will not encode the new field. This leads the receiver to fail decoding the message when trying to decode the length of the variable length field.

Is there a best practice to avoid this situation other than package versioning? Should I add a build-time check asserting that the message version used in package 2 is 0, so that I don't end up with package 1 ahead of package 2?

[donmendelson]

The SBE v1.0 specification states:

The <messageSchema> root element contains a version number attribute. By default, version is zero, the initial version of a message schema. Each time a message schema is changed, the version number is incremented.

Version applies to the schema as a whole, not to individual elements. Version is sent in the message header so the consumer can determine which version of the message schema was used to encode the message.

In short, you cannot mix versions within a schema. However, it is generally valid to use more than one SBE schema over a session. The receiver would be able to read the schema, template, and version numbers in each message header and therefore, know which decoder to use.