centiservice / mats3

Mats3: Message-based Asynchronous Transactional Staged Stateless Services
https://mats3.io/
Other
63 stars 7 forks source link

Hand-crafted binary serialization and deserialization of MatsTrace (or "direct lookup" into byte array) #6

Open stolsvik opened 6 years ago

stolsvik commented 6 years ago

After refactoring the MatsSerializer to be binary (byte[]), it has occurred to me that it would probably be both denser and faster to use hand-crafted serialization and deserialization. This would be OK since the MatsTrace will be very slow-changing after it has stabilized.

Maybe include a version number as the first byte or something like that, so that it could evolve to support optional features. E.g. if added later, a "context.getInitiatedDate()" could return 0 if the underlying MatsTrace was on an old version, and then return the java timestamp if it was supported.

Consider implementing this by a MatsTrace backed directly by a byte-array: Upon createNewMatsTrace, a new byte-array was created, filling in information directly into the array. Getters would directly "substring" into the byte array to get stuff out. Modifiers would use "binary patching" when creating a new MatsTrace - i.e. when doing addRequestCall, the resulting MatsTrace was directly created into a new byte-array with the new piece patched in. This would hopefully make the "serialization" and "deserialization" be very quick, as it would pretty much not happen..! (Notice that this only regards the MatsTrace "container" - the DTOs and STOs are a different part of the protocol, and would use Jackson, Jackson with protobuf-backend, or Protobuf directly - or whatever.

stolsvik commented 6 years ago

Judging by serialization/deserialization times, and compress/decompress times (once I made these timings available!), this won't shave more than at most a few milliseconds off the total processing time for a send/receive - and that is for large messages. For small messages, i.e. the absolutely most common, it would shave fractions of milliseconds.

stolsvik commented 6 years ago

Interesting blog post about UTF-8 encoding: http://psy-lob-saw.blogspot.no/2012/12/encode-utf-8-string-to-bytebuffer-faster.html