search

trustoverip / tswg-cesr-specification

Composable Event Streaming Representation (CESR) Specification
https://trustoverip.github.io/tswg-cesr-specification/
Other
6 stars 8 forks source link

Count Code tables have "quadlets/triplets" for nearly all count codes even for codes that should denote counts #86

Closed daidoji closed 6 months ago

daidoji commented 6 months ago

Its clear from the reading and keripy that some count codes count things and some codes fit into the TLV scheme to count bytes (via the quadlet/triplet inference depending on what domain you're in). However, in the tables in the annex, nearly all count codes are listed as counts of quadlets/triplets. This is quite confusing for the implementer.

These tables should be updated to clearly reference whether the count code is counting primitives that will follow in the stream or if its count is counting quadlets/triplets.

SmithSamuelM commented 6 months ago

Yes in the version 1 of CESR count codes could either count primitives or nested groups or count quadlets/triples of B64 chars/B2 Bytes respectively. In Version 2 this has been fixed. All count codes only count quadlets/triplets not primitives/groups. So examples that use version 1 count codes need to be updated to version 2. Since the spec is version 2 and version 1 is deprecated.

A nuance not obvious, is that since the only place version 1 of KERI uses cout codes is in attachments which are unsigned and the LMDB database does not store the count code only the embedded primitives then going forward there should be no reason once everyone upgrades to version 2 code to ever use version 1 count codes again.

To clarify. when replaying a KEL the atttachments can be upgraded on the next replay to use version 2 CESR count codes since they are not signed. The events themselves are signed so can't be upgraded, but because the events themselves in version 1 only use primitives, no count codes, then nothing breaks. The JSON, CBOR, and MGPK still works. So we can simply update any examples of version 1 key event messages to use version2 CESR count codes in the attachments and they will be all good. The primitive codes inside the messages don't change. And version 2 messages change the field composition (ile labels) that appear in the Message.

Once we start using CESR native events which are signed, those CESR native events only use version 2 count codes.

So we have a clear upgrade path in code.

  1. upgrade all implementations to use CESR v 2 for countcodes used for attachments and CESR version 2 for primitives. Because CESR v2 primitives are backwards compatible. Nothing breaks if both sides are using CESR v2. The KERI version can be either v1 or V2. Everybody just uses CESR V2 for attachments.
  2. Upgrade KERI version to V2. This requires CESR v2. Since v1 KERI is still supported old KELs still are verifiable but with CESR v2 attachments. New KERI V2 messages may now use CESR native serialization using CESR v2.
SmithSamuelM commented 6 months ago

Since we want to eventually before handing off to ISO fix all the examples to be working examples, this upgrade of attachements to version 2 CESR can happen at the same time. This requires getting the python reference implementation of CESR v2 working first. Alternatively, if one wants to manually change the count codes to v2 ones without actually putting in the right counts then that might be a stop gap

daidoji commented 6 months ago

Oh cool, then I will probably close this issue as it isn't an issue for v2.

If we build an implementation of KERI V2/CESR V2 and we receive KERI V1/CESR V1 over the wire, the primitives are the same but wouldn't we still need the current table's semantics to be able to parse those primitives? Or is your advice just to grab those semantics now and replace with V2 whenever we come across at some point deprecating support for V1 altogether?

SmithSamuelM commented 6 months ago

the new Counter class which is in development (has not yet replaced the old Counter class) has both tables v1 and v2 that are selected based on a version passed in to the Counter. So it will keep the V1 semantics but we want to deprecate that quickly. Given that there are only at most a handful of v1 implementations. We should be able to then at some time remove the v1 tables.