ndsev / zserio

zero sugar, zero fat, zero serialization overhead
https://zserio.org/
BSD 3-Clause "New" or "Revised" License
109 stars 27 forks source link

Emit Source Positions for Compounds #652

Open johannes-wolf opened 3 months ago

johannes-wolf commented 3 months ago

Is your feature request related to a problem? Please describe. For mapping back to blob-positions, we need an option to get the blob-position an object (compound/struct) was read from. This is specifically needed for compound types only.

Describe the solution you'd like The idea is to add a new zserio C++ generator option (e.g. -withSourcePosition) that returns an object's blob-position in bits.

Describe alternatives you've considered Calculating the offsets by summing the results of getBitSize() is not feasible due to potential complexities involved.

Additional context A working implementation can be found in PR #648.

mikir commented 2 months ago

The current solution introduces new command line argument -withBitPositionCode. This command line argument enables generating of the bitPosition() method for all Structure, Choice and Union types. The bitPosition() method returns the bit position in the parsed blob after reading.

Note that the returned bit position is valid only directly after read! If the Zserio object has been changed after reading, the result is unspecified!

Because of the note above, better solution would be to generate bitPosition() method only for immutable objects (using withoutWriterCode option). However, this solution does not address the user's use case.

Another solution could be to overload reader constructor to introduce observer which will be called during parsing. This observer would get reference to the Zserio object together with its bit position in the blob. Such solution would allow application to implement whatever it wants. However, this solution would require bigger effort on both sides.

Because of that, the current solution, which can be easily misused, has been chosen. However, it will be clearly stated in the Zserio User Guide that this feature is experimental, is not part of API, and can be removed without any warning.

mikir commented 1 month ago

There is a new request not to get only bit positions but bit sizes as well. This is because it is very difficult to get correct bit sizes for packed compounds.

We suggest to redesing the current solution in the following way:

This will allow to add more parsing information in the future without any change of the design.