Miosss
commented
9 years ago @MikeFair So, in the beginning, what are the differences between this and current STC (with possibility to repeat header)? Apart from slightly different markup? I do not remember now where it was, but we had long discussion about only repeatable headers - but they were dropped for now. I remember that Steve objected very much about the idea to use STC to define containers of mixed types (and for that mostly repeatables were discussed, and this is main point of your proposal I think).
Since, the only meaningful interpretation of this is
Well no, decoder should guess what is proper in this situation? Sounds like a bad idea.
There really isnt much diference between writing [#][128][d]... and [[][#][128][d]...[]]
MikeFair
lightmare
This proposes using [#] to create a repeating 'Count' of values for a TLV, a CTLV where C is optional and defaults to 1.
Whereas TLV = a single Value of the given type CTLV = C (Count) values of the given type
This isn't much different from what already exists, it's just redefining [#] in a way that I think makes it clearer and more concise about what it can do and how it modifies the UBJ stream.
It's not saying anything about whether or not [#] creates an array container or not, it's simply instructing the parser to retrieve [COUNT] values of [T] instead of [1] value of [T].
This brings this proposal very close to the "repeatable headers" proposal but uses a different style of defining [#]. In this proposal [#] is modifying a TLV to change the default count of values of T to retrieve from 1 to [COUNT]. As such it is restricting [#] to only ever modify 1 type, not create a repeating sequence of types as in #50; nor does it modify the array or object container like #51 does.
We want to restrict [#] to a single type because that provides the most optimization, with the most consistency, for the least amount of complexity. There are many parser optimizations available when it's known that a continuous region of the UBJ stream are all of the same [T].
This proposal, like the repeatable header idea, preserves the "Mixed-Type" possibilities of arrays while also providing all the benefits of singly typed arrays.
For example:
[{] // Open an object [3][foo][[] // start an array for "foo's" value [#][250][d].....250 [d] values ...... // put 250 float32 values in it []] // close the array [}] // Close the objectInstead of attempting to define a float32 array, it defines an array of solely float32 values. If there was more than one section within the area, this doesn't interfere with that.
[{] // Open an object [3][foo][[] // start an array for "foo's" value [#][250][d].....250 [d] values ...... // put 250 float32 values in it [#][250][d].....250 [d] values ...... // put 250 float32 values in it [#][250][d].....250 [d] values ...... // put 250 float32 values in it [#][250][d].....250 [d] values ...... // put 250 float32 values in it []] // close the array [}] // Close the objectFor some reason the encoder on this array decided to provide 4 sections of 250 elements each. Perhaps it's streaming the results and it didn't actually know how many [d] values there would be in advance so it wrote them out in sections like that.
Now, let's say, as is legal, there are some "Inf" and "-Inf" strings in the middle of the series. If we attempted to define this array as a float32 array, then we'd be out of luck. But because we aren't constraining the type of the array at all, merely providing the values of the array in fixed type sections, then this isn't a problem.
[{] // Open an object [3][foo][[] // start an array [#][250][d].....250 [d] values ...... // put 250 float32 values in it [#][249][d].....249 [d] values ...... // put 249 float32 values in it [S][4][-Inf] [#][250][d].....250 [d] values ...... // put 250 float32 values in it [#][249][d].....249 [d] values ...... // put 249 float32 values in it [S][3][Inf] []] // close the array [{] // Close the objectWhile this shares a lot with the repeatable headers approach, it keeps the context [#] constrained to just a single type.
In closing, I'll just add that [#] when defined this way can also be used as a value outside of an array, for instance, as an object field's value.
[{] // Open an object [3][foo][#][128][D]........ 128 [D] Values ......... // "foo": [ ... 128 [D] Values .... ] [}] // Close the objectThis defines the object with the single key foo. Where it's value belongs it uses the CTLV format to define 128 float64 values. Since, the only meaningful interpretation of this is an array of 128 float64 values, it can define the entire array right there on the spot and avoid the additional [[] and []] markers.
If the data supports this syntax because it truly is singularly typed then this is easily handled. If on the other hand, the stream contained any "Inf" or "-Inf" strings it would not be able to use this format and would use the [[] ... mixture of [d] values and [S] values in an array ... []]