Direct offsets for variable length fields

[adkapur]

We have a requirement in which a wrapper message is composed of a number of sub messages or templates with each of these representing specific sections of data, for example:

|--------------message-----------------------------------------------| |---common---||---specific message---||---metadata--|

Common template represents data which is common across all messages, specific message refers to NewOrderSingle, OrderCancelReplaceRequest etc and metadata refers to internal system data

This way if changes are made to a specific template such as metadata then the impact could be isolated to consumers of that section only

If we could put offset fields in the “wrapper message” that point to the start of the individual sub-messages then consumers could jump straight to that particular sub-message. We could also perhaps include the total length as well of each sub message

Could this information be placed in the root block of the schema not in the message header necessarily but in a separate index like construct?

[billsegall]

This is something that I think is worthwhile.

It would also be desirable to be able to reset a group to it's starting point so you could for example iterate over it multiple times.

Unfortunately the code generators are not really set up for this model so it would be a not insubstantial patchset.

[donmendelson]

@billsegall, ability to re-parse a message or group is an implementation issue. There is nothing in the SBE specification that prevents that.

[billsegall]

I agree, This is not a spec issue