Protocol Buffers (a.k.a., protobuf) are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data. You can find protobuf's documentation on the Google Developers site.
Parsing protobufs depends on this custom stream tokenizer.
Why it needs a custom tokenizer and why it is the way it is is described in detail in this comment.
The tokenizer is not the most interesting part of Protobuf, but the rest of the code seems either overwhelmingly complex, or impossible to reason about in isolation, usually both.
General
Description
Protocol Buffers (a.k.a., protobuf) are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data. You can find protobuf's documentation on the Google Developers site.
Parsing protobufs depends on this custom stream tokenizer.
Why it needs a custom tokenizer and why it is the way it is is described in detail in this comment.
Links
What makes it interesting
Related work
No.
Other
The tokenizer is not the most interesting part of Protobuf, but the rest of the code seems either overwhelmingly complex, or impossible to reason about in isolation, usually both.