see also Coze

[zamicol]

Hello!

I noticed you implemented jsf. Impressive! I just became aware of jsf and was poking around the web to see who else knew that jsf was a thing.

As you're probably interested in this sort of thing, I wanted to reach out and see if you're interested in a related project: https://github.com/Cyphrme/Coze

We're always looking for feedback and help.

Have a good one!

[yurikhan]

Hm. I completely forgot I did this. These days, I have the impression that JSF superseded by JWS/CT; while an expired ID, it may be the closest to an IETF standard that we have now.

I looked at the Coze readme right now and I don’t see a Prior Art section explaining which problems it solves that are not addressed by competing schemas.

[zamicol]

Thanks yuikhan for the reply!

Next time I push to the main Coze README I'll add to this to this FAQ header How does Coze compare with prior arts? so it's easy to find with a control+f

See the Google slides for comparison with other works. At the moment, the "Coze Vs. Others" starts on slide 88.

There are additional documents in the CozeX repo In particular, see the "Coze Vs" document for comparison with similar technologies.

A lot of the documents are a mess and need further revision. A lot of the documents are more like notes than something formally published.

[yurikhan]

Slides are not publicly accessible, and “Coze Vs” mentions neither JSF nor JWS/CT. Both efforts are by Anders Rundgren, and both address the same niche as Coze as far as I understand it — keeping JSON human-readable while at the same time making it signable and signature-verifiable by defining a canonicalization algorithm.

You really need to formalize your spec. As it is, by reading the JWS/CT RFC draft, I can readily convince myself that it is viable. That is, signing a JSON message, then adding/removing insignificant whitespace, rearranging object properties order at any depth, changing strings between literal UTF-8 and \u-escaped forms, adding redundant trailing zeroes after the decimal point in number values, still keeps the signature valid. Reading the Coze main README, specifically Canon, I only see explicitly specified protection against whitespace differences and top-level property order changes.

Using the online demo, if I do:

1. Generate a key pair. I get a public key:

   {
    "alg": "ES256",
    "iat": 1696838831,
    "kid": "My Cyphr.me Key.",
    "tmb": "hxHjARNpIZBumXMBzEBdsv4G9DL_9KIy2Ig8bED8ras",
    "x": "HjcN5bPLITlMy7ZBHAEZhfPIoe1p7IMSUr74cxkUD-G3BUnTX2coXyBinqHySmUjI5kYIZ2cDXcryNJqCp_SiQ"
   }

   and a private key:

   {
    "alg": "ES256",
    "iat": 1696838831,
    "kid": "My Cyphr.me Key.",
    "tmb": "hxHjARNpIZBumXMBzEBdsv4G9DL_9KIy2Ig8bED8ras",
    "x": "HjcN5bPLITlMy7ZBHAEZhfPIoe1p7IMSUr74cxkUD-G3BUnTX2coXyBinqHySmUjI5kYIZ2cDXcryNJqCp_SiQ",
    "d": "ECI1L1j2ilayWBGHk-DRyopVUrH1R221b-u85h2c7_o"
   }

2. Sign the message {"xyzzy": {"foo": "bar", ""}}. I get

   {
    "pay": {
      "xyzzy": {
        "foo": "bar",
        "baz": "quux"
      }
    },
    "sig": "qHuiqxnRURAdknW_KQ0yzIbu1fd8Msp1DEsp8N0EBR4MPWjVOpzYKuAhqUnhRkNad9N03pbI31qTf937FE1lNA"
   }

3. Copy and paste this signed message unchanged into the input box, click Verify. It succeeds.

4. Change the order of foo and baz within the pay.xyzzy subobject:

   {
    "pay": {
      "xyzzy": {
        "baz": "quux",
        "foo": "bar"
      }
    },
    "sig": "qHuiqxnRURAdknW_KQ0yzIbu1fd8Msp1DEsp8N0EBR4MPWjVOpzYKuAhqUnhRkNad9N03pbI31qTf937FE1lNA"
   }

   Click Verify. It says “unable to verify” which I interpret as a failure.

I consider preservation of signature validity under re-serialization of JSON by any naïve tool an essential property of a JSON signing algorithm whose output format is also JSON, and that part is really well thought out in JSF and JWS/CT.

[zamicol]

yurikhan, thank you for the thoughtful reply.

Slides are not publicly accessible

Fixed! I've been organizing documents and I think the permissions got changed. The entire slides are currently about ~160 and the "Coze Vs. Others" starts on slide 88

JSF

I just became aware of JSF and I'm still getting up to speed with it. We made Coze while unaware of JSF, so it's thrilling to compare notes! Using now for a timestamp field is brilliant and so far that's been my favorite aspect of JSF.

which I interpret as a failure

That example is working as Coze is written. Coze verifies and signs UTF-8 bytes first, before being parsed as JSON. The field pay is labeled, meaning any naïve tool that is doing manipulation needs to interpret pay first. We felt that this was enough of a guard, however, if naïve tool manipulation is a common issue even with the guard, I would like to have this concern better addressed.

You've concerns of representation touched on something that we debated for a long time: Coze's canonicalization method. We gave a lot of considering to doing deep Unicode ordering, in fact, that was the original design of Coze. However, when manipulating large payloads with lots of fields, fields that were more relevant to human readers would get buried. So we decided to allow any ordering and that made it feel much more ergonomic. Important fields could be at the top, less important buried at the bottom.

As far as normalization, In Coze the digest is over bytes. Coze signs and verifies digests, and that digest can be of anything.For JSON this is UTF-8 first and then JSON. A message can be verified by Coze and be represented by various systems differently. Coze just has some addition rules for messages that are JSON. As far as \u-escaped forms, I think if Coze is taking the position that duplicates are bad and a security issue, it also needs to address escapes. I think OLPC's makes the most sense.

I am thinking about adding a stricter JSON canonicalization to CozeX (Coze eXtended), which yes would address representation as well. That's a rabbit hole I'm currently researching. Coze chose its current canonicalization exactly because I wanted to avoid writing extensive representation rules, which are much more extensive than what it currently requires. Any stricter canonicalization would probably be added to CozeX, unless there was a really good reason to add it to core, the main Coze spec.

• Zach

[yurikhan]

when manipulating large payloads with lots of fields, fields that were more relevant to human readers would get buried

This is an issue that resonates with me. I, too, very much prefer to write human-readable JSON with more important and/or identifying field first.

However, this does not have to be reflected in the canonical representation used for signing. After all, that representation strips all syntactic whitespace, which makes it human-unreadable.

So I think an ergonomic application using signed JSON under the hood will:

• On the user-facing end:
  • Either preserve the ordering as authored by the user, or define a canonical ordering for each object type, at any depth.
  • It may also impose some ordering on array elements, something that no general-purpose JSON canonicalization scheme can do.
  • It will either retain whitespace as authored or use any reasonable pretty-printing algorithm.
• For signing, re-serialize into the canonical form prescribed by the signing specification.
• After calculating the signature, plug it into the user-facing represenation and send the result through the network or save it to disk.
• Before verification, pluck the signature (and any non-significant fields) from the user-facing represenation, re-serialize the result into the canonical form, and verify the signature.

With JWS/CT the above is viable. With Coze, serialization of the nested objects is underspecified so changing the properties’ order invalidates the signature.

I think OLPC's makes the most sense.

By disallowing floating point numbers, it makes its job much easier, but limits the applicability of any further specification based on that.

JWS/CT bases itself on JCS (RFC 8785) which closely corresponds with the Javascript’s JSON serialization algorithm and does not place any artificial restrictions on the JSON content.

By being an RFC, JCS has the benefit of having been reviewed by IETF, and high chances of interoperable implementations in various languages.

[zamicol]

I think OLPC's makes the most sense.

In the narrow sense of mandatory escapes. I think there's only really three choices here: 1. Only minimal escapes (" and /), 2. escape all non-printable characters, and 3. stick with the RFC's 34 mandatory escapes.

For 1: OLPC chose 1, and requires only the minimal escapes, " and \. This makes a lot of sense, especially since the RFC (all four revisions 8259, 7159, 7158, and the original 4627) failed to define all non-printable or even all control characters. I would argue just the two as mandatory escapes is in alignment with the original json.org spec. \u0000 to \u001F is an addition by Crockford's original RFC. This is also the smallest form a JSON message can be, all other options further bloat messages.

For 2: It would have been nice to see a standard define all Unicode non-printable characters, but I'm not even sure this is practical or possible.

For 3: The last option is just to stick with the RFC, the 34 mandatory escapes defined by 8259. Having some non-printable code points escaped while leaving other non-escaped is inconsistent and not very helpful. Sure, it might be nice in narrow use cases and that's what the original thought was, it just wasn't thought through, for example it forgot DEL which is in the ASCII range. Old school ASCII doesn't carry over to Unicode, but it didn't even implement ASCII correctly. It's a common mistake, it's just unfortunate to have it codified.

Either preserve the ordering as authored by the user, or define a canonical ordering for each object type, at any depth.

Coze right now has an implicit canon for pay that can be explicitly denoted by can. "The canon of pay is the currently present fields in order of appearance." I think that is reasonably interpreted as covering deep canons.

However, we've not implemented deep canons, but it wouldn't be difficult. For your example, [{"xyzzy":["foo","baz"]}] is the canon.

pretty-printing algorithm

The tool has pretty/compact options under "advanced".

Here's a link for your example that triggers "pretty": https://cyphr.me/coze#?input={%22pay%22:{%22xyzzy%22:{%22foo%22:%22bar%22,%22baz%22:%22quux%22}},%22sig%22:%22qHuiqxnRURAdknW_KQ0yzIbu1fd8Msp1DEsp8N0EBR4MPWjVOpzYKuAhqUnhRkNad9N03pbI31qTf937FE1lNA%22}&dontSignRevoke&updateIat&key={%22alg%22:%22ES256%22,%22iat%22:1696838831,%22kid%22:%22My%20Cyphr.me%20Key.%22,%22tmb%22:%22hxHjARNpIZBumXMBzEBdsv4G9DL_9KIy2Ig8bED8ras%22,%22x%22:%22HjcN5bPLITlMy7ZBHAEZhfPIoe1p7IMSUr74cxkUD-G3BUnTX2coXyBinqHySmUjI5kYIZ2cDXcryNJqCp_SiQ%22,%22d%22:%22ECI1L1j2ilayWBGHk-DRyopVUrH1R221b-u85h2c7_o%22}&selectedAlg=ES256&import&pretty&advanced&private