Internet of Things data protocol - simplistic text and CSV-friendly IoT data protocol. Options: Schema-less, schema+versioning, building deltas for data values. Easy to re-use.
Apache License 2.0
0
stars
2
forks
source link
Schema + versioning concept for IoText data protocol #2
It will be more optimal - and of course it is prelude to Change Data Capture technique - to add an implementation of schema versioning with indexes instead of repeated metrics names like this:
as you can see, we have a bigger S item code/kind for schema definition and all repeated metrics names are indexed by hexadecimal 3 positional digits. It gives us ~4k metrics in one row/message.
With schema + versioning, we can save approx. 50% of message size. Much more, compared to JSON formated the same message!
In general with the schema-less basic version of the protocol we can send data like this:
It will be more optimal - and of course it is prelude to Change Data Capture technique - to add an implementation of schema versioning with indexes instead of repeated metrics names like this:
as you can see, we have a special item
s|v1- information about versioning and version name.At the same time schema message needs to be sent to the server/receiver, too - schema format proposal:
as you can see, we have a bigger
Sitem code/kind for schema definition and all repeated metrics names are indexed by hexadecimal 3 positional digits. It gives us ~4k metrics in one row/message.With schema + versioning, we can save approx. 50% of message size. Much more, compared to JSON formated the same message!