There are a few cases (in particular, headers) where the length of the data
is maintained in the serialized data. This is a problem for several reasons. First,
it requires you to know the serialization format to decode the length. Second,
it presumes that the serialization format encodes the length. Third, compression
makes it impossible to know how many bytes to read.
Currently, all data already includes explicit lengths in the header. But what doesn't
include an explicit length is the header itself. So this needs to be changed so there
is an explicit header length in the format. Then all reads can be done for precisely
the required number of bytes.
This will be necessary to address #11 but it will also clean up the APIs for the
serializers significantly and shouldn't impact the read count.
There are a few cases (in particular, headers) where the length of the data is maintained in the serialized data. This is a problem for several reasons. First, it requires you to know the serialization format to decode the length. Second, it presumes that the serialization format encodes the length. Third, compression makes it impossible to know how many bytes to read.
Currently, all data already includes explicit lengths in the header. But what doesn't include an explicit length is the header itself. So this needs to be changed so there is an explicit header length in the format. Then all reads can be done for precisely the required number of bytes.
This will be necessary to address #11 but it will also clean up the APIs for the serializers significantly and shouldn't impact the read count.