gritzko / ron

(dated, see the site) Replicated Object Notation, a distributed live data format, golang/ragel lib
http://replicated.cc
Apache License 2.0
360 stars 7 forks source link

add meat to the binary spec #9

Closed gritzko closed 6 years ago

gritzko commented 6 years ago

feedback requested @cblp @archagon @olebedev see https://github.com/gritzko/ron/blob/binary/README.md#wire-format-binary

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 3 unresolved discussions.


README.md, line 324 at r1 (raw file):


Frame descriptor type bits encode major/minor versions of the protocol (two
m.s. bits for the major, other two for the minor, e.g. `10 00` for 2.0). 

Decipher "m. s." please. It doesn't look like a widespread abbreviation https://en.wikipedia.org/wiki/Most_significant_bit


README.md, line 345 at r1 (raw file):

3. `11` Atom
    * `1100` value UUID, uncompressed
    * `1101` integer (big-endian int64)

signed?


README.md, line 346 at r1 (raw file):

    * `1100` value UUID, uncompressed
    * `1101` integer (big-endian int64)
    * `1110` string (...)

...?


README.md, line 352 at r1 (raw file):

field or op in question.  The length value given by a descriptor does not
include the length of the descriptor itself.  Hence, the *cited* length is less
that the actual byte length by at least 1 byte.  Field lengths are "nested"

"less than"


README.md, line 357 at r1 (raw file):


If a field or a frame is 0 to 7 bytes long then it has its length coded
directly in those four bits (m.s. bit being set to 0). If the m.s. bit is set

MSB of the "length" field or the descriptor? Why not just "length flag bit" or "bit 3"?


README.md, line 358 at r1 (raw file):

If a field or a frame is 0 to 7 bytes long then it has its length coded
directly in those four bits (m.s. bit being set to 0). If the m.s. bit is set
to 1, then the other three bits code the byte length of the following

"encode"?


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 9 unresolved discussions.


README.md, line 361 at r1 (raw file):

big-endian number coding the actual field length (1 to 7 bytes).

Consider a time value request frame: `*now?.`

What is a "request"? Requests/queries are not described (in "Formal model" section or anywhere).


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 10 unresolved discussions.


README.md, line 367 at r1 (raw file):

* op descriptor: 1 byte (cited length 4, `0011 0100`)
* uncompressed UUID descriptor: 1 byte (cited length 3, `0100 0011`)
* `now` RON UUID: 3 bytes (`0000 1100  1011 0011  1110 1100`)

Where did it come from? How can uncompressed UUID be 3 bytes length if it always is 16 bytes?


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 11 unresolved discussions.


README.md, line 371 at r1 (raw file):

As UUID length is up to 16 bytes, UUID fields never use a separate length
number. UUID descriptors are always 1 byte long. The length flag is treated
like the other three bits. 

"The length flag bit along with 3 length bits is treated as a 4-bit unsigned integer."


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 12 unresolved discussions.


README.md, line 376 at r1 (raw file):


* for integer/float atoms, length 0 stands for zero value (`=0`, `^0.0` resp),
* for strings, that stands for an empty string,

BTW you may add a note: "this is not special, actually"


README.md, line 377 at r1 (raw file):

* for integer/float atoms, length 0 stands for zero value (`=0`, `^0.0` resp),
* for strings, that stands for an empty string,
* for frames, length 0 stands for an empty frame (e.g. a keepalive),

BTW you may add a note: "this is not special, actually"


README.md, line 381 at r1 (raw file):

* for uncompressed UUIDs, 0 stands for length 16 (full-length UUID),
* for ops, length 0 stands for a no-fields op (e.g. if using the `?!`
  construct, the second op has all UUIDs skipped).

What is ?! construct? Where is it described?


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 15 unresolved discussions.


README.md, line 398 at r1 (raw file):


Namely, UUIDs 1..8 bytes long have the *origin* part set to zeros (all 8 bytes)
and the least significant bytes of the value also set to zeroes.

least?


README.md, line 399 at r1 (raw file):

Namely, UUIDs 1..8 bytes long have the *origin* part set to zeros (all 8 bytes)
and the least significant bytes of the value also set to zeroes.
These are often-used "transcendent" name UUIDs (`lww`, `rga`, `db`, `now`, etc).

What are their exact UUID values?


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 17 unresolved discussions.


README.md, line 367 at r1 (raw file):

Previously, cblp (Yuriy Syrovetskiy) wrote…
Where did it come from? How can uncompressed UUID be 3 bytes length if it always is 16 bytes?

OK, I've read the "Uncompressed UUIDs" section. Please refer to it here.


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 17 unresolved discussions.


README.md, line 409 at r1 (raw file):

Zipped UUIDs are serialized as deltas to similar past UUIDs.  That provides
significant savings when UUIDs come from the same source (same origin bytes) or
have close timestamp values.  Repeat UUIDs are simply skipped, same as in the

"Repeated"


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 18 unresolved discussions.


README.md, line 412 at r1 (raw file):

Base64 notation.

The origin value is either reused in full or rewritten in full. That is decided

"origin part"?


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 19 unresolved discussions.


README.md, line 419 at r1 (raw file):

compressed relative to the same kind of UUID in the previous op (e.g. event id
relative to the previous event id).  In the long mode, an UUID is compressed
relative to a past uncompressed UUID.  A decoder must remember 16 last

The encoder must keep track of them, too.


Comments from Reviewable

cblp commented 6 years ago

Review status: 0 of 1 files reviewed at latest revision, 20 unresolved discussions.


README.md, line 447 at r1 (raw file):

+----+----+----+----+----+----+----+----+
|  1 |zero tail len | past uncompressed |
|    |  (half-bytes)|   UUID index      |

Is index counted from the current position towards past, or from -16 towards future?


README.md, line 463 at r1 (raw file):

Integers are serialized using the zig-zag coding (the l.s. bit conveys the sign).

Floats are serialized as ISO floats (4-byte and 8-byte support is required,

It is approved by ISO, and IEC, but is most widely known as IEEE 754.


Comments from Reviewable

cblp commented 6 years ago

Reviewed 1 of 1 files at r1. Review status: all files reviewed at latest revision, 22 unresolved discussions.


Comments from Reviewable

cblp commented 6 years ago

Reviewed 1 of 1 files at r2. Review status: all files reviewed at latest revision, all discussions resolved.


README.md, line 114 at r2 (raw file):

2. An op is an immutable atomic unit of data change.  An op is a tuple of four
"key" [UUIDs](uuid.md) and zero or more "value" *atoms*. An op goes under one
of four *terms*.

What does "goes under" mean? What is "term"? Is it a section in a frame? Is it a type of op?


Comments from Reviewable

cblp commented 6 years ago

Reviewed 1 of 1 files at r3. Review status: all files reviewed at latest revision, 2 unresolved discussions.


Comments from Reviewable

cblp commented 6 years ago

Review status: all files reviewed at latest revision, 2 unresolved discussions.


README.md, line 384 at r3 (raw file):

number. UUID descriptors are always 1 byte long. The length flag bit joins the
other three length bits to code 16 length values (1 to 16, 0 stands for 16). 

We found that Op must skip length field because it is already limited by the frame and variable-length length field interferes with streaming encoding.


Comments from Reviewable

cblp commented 6 years ago

Пора сливать уже.

cblp commented 6 years ago

(Сливать в хорошем смысле)

gritzko commented 6 years ago

Я сомневался, как это всё повлияет на поломанные фреймы. Никак не повлияет, и хорошо.

cblp commented 6 years ago

поломанные — это какие?