RFC 0021 - Encoding byte for content type of envelope and content messages

[tplooker]

Currently as the RFC 0021 states, we are dependent on a JOSE based format for our envelope level and a JSON based structure for our content level message.

I would like to informally surface the idea to gauge community support that we should have an encoding byte for both of these message levels that defines the expected content type of the message.

The behavior would involve the first byte of a message (both envelope and content levels) defining the encoding definition. For example x01 = JOSE format for an envelope level message.

Rational

For did communications to be a future proof protocol I believe it should not have a dependency on a particular serialization format, instead did communications should merely define a data model and a set of supported serialization formats. Delaying the addition of support for this will make it harder to add in the future, therefore I believe it is important to have this discussion in near term.

Considerations

With both message levels, adding this encoding type opens a new avenue for negotiation, which will increase complexity for parties using did communications.

[tplooker]

Subsequent to this I feel the emerging specs with CWT RFC 8392 and COSE RFC 8152 would be an example of formats to support as alternatives to the current JOSE based object formats. Thoughts @dhh1128 @kdenhartog @TelegramSam

[dhh1128]

Can we just have the encoding byte be { for JOSE? That would avoid the need for another byte besides the one that already begins our JOSE messages.

I am not opposed to having a different format, but I AM opposed to having multiple formats that exist in free variation. That is, I don't want to simultaneously be supporting JOSE-based and non-JOSE-based serialization. It's too much complexity for not a lot of incremental benefit. I want to pick one and stick with it. So I would view this serialization detection as a way to tell whether the data is in the old format or the new format, not as a way to pick between equally supported alternatives.

Also, although I am not opposed to a non-JSON format for envelopes, I am opposed to a non-JSON format for plaintext. I don't see a compelling benefit to that. It seems to create alternatives for no good reason.

[kdenhartog]

Taking note that I saw this, but want to think some more on this before offering a stronger opinion. Initially I was in favor of supporting other serialization formation. I found the additional benefits of supporting multiple types at the content layer beneficial, but I find @dhh1128 argument compelling.

With regards to the serialization format I think this is the sticking point that holds me from affirming support of the cryptographic envelope supporting multiple formats right now. I've thought using the transport layer may be a good idea, but I'm hesitant that it's a conflation of layers. I'd much rather prefer that it was self contained within the envelope, but this may be hard if not impossible to do. The { seems like it could work, but I'd have to investigate more.

More to come on this later.

[TelegramSam]

I'm also a fan of looking at the first byte of different formats. If { means json, then we can inspect the parsed json to figure out anything else relevant. Any other format such as cbor should have a predictable first byte.

[tplooker]

Personally I think the { could be a weak convention to use for encoding and a future valid binary format could easily collide with this.

[TelegramSam]

A future valid binary format could avoid this on purpose.

For reference, here are the byte encodings for msgpack (map): https://github.com/msgpack/msgpack/blob/master/spec.md#map-format-family

and cbor major type 5: https://tools.ietf.org/html/rfc7049#section-2.1

[tplooker]

@TelegramSam @dhh1128 @kdenhartog, I would have preferred an explicit encoding character but, I'm happy to close this issue in favor of the simpler solution suggested.

In order to disambiguate DIDComm messages (essentially just a JOSE structure) from other valid JOSE structures I would like to suggest we add an identifier to the header I believe @llorllale @troyronda and team are already doing this in their implementation? E.g the following example.

{
  "protected": {
  "alg": "ECDH-SS+XC20PKW",
  "enc": "XC20P",
  "typ": "prs.hyperledger.aries-auth-message" //This identifies this is a DIDComm message
  },
  "recipients": [
       {
          "encrypted_key": "C1ZgZxgIKne86FlGuct0L2y2UENtHyOS78KvoVgBAaM=",
          "header": {
                "apu": "dpQnrrNMi8CHC1sRLxcKdzHUfYqM9hMI4P8VpLMArVYdJQlmnIzF6niGjzTCNF6UBobaTmm6msTlEK2chntsOA==",
                "iv": "Po7Wi7CKWHeCof450oyrFB9A_x8D8LQa",
                "tag": "QkkS6QqQLaWRJDtEK0htKQ==",
                "kid": "ETu2KkCZUawEHq99z2RubyCsnVyaLhzMiYTHMu3uVncp",
                "oid": "PBqOfOOkMM3hDUxSunySlld6-15qdbMdWajolmmKVYs0bog785bnqckvoPYDkqoMXwVPtr-qPQgIpg9A"
            }
       },
       {
          "encrypted_key": "pGL8MBT1WUXjfXuwSl2enj5N1BLtmeEZ1wAZAzUoH60=",
          "header": {
                "apu": "iSa2FM-_nOzxG4IjX5MOUv1uvEFs-nA-fYlrLry66VVrNJxakXclPC7UnJ_RtqI_cwlEgJ_GaVHcNv84NA8VjQ==",
                "iv": "xSEHpsn1iuy4x3KTfeHdcdBRziqCvImG",
                "tag": "vsU_H6K5m4_jVFWNxxV--A==",
                "kid": "BkcdRt9AChfFXGvKU3ecjzyNCzoP4EBVvm5dtgvqv5UX",
                "oid": "u1PbWv2wIIV4BkOibItT4DzkzVb6d75hLhS01Ha1CzapUX5xjBlcyjwrl6nIOrARBvoJyH0FXZdkxec7"
            }
       },
       {
          "encrypted_key": "PsFDcK7-VHzH-E-KDC-Cy1NHZYKepv1X7zolbuRbdbw=",
          "header": {
                "apu": "PAYUGCNWRKGq82HvOxyMVE6VI13mYlvowK5X3NqZsFiwqpTTwXgNTZA5Tnxch8vsBZAzeEpmEvI3M9YpJflnrA==",
                "iv": "s_FfFvulVhFKwCoalqp9qESsEtWnDgZ9",
                "tag": "7IIQuM9Hs2Zqsj9Vga2nAQ==",
                "kid": "EikHy4Hmbw5zGUMPGYDoAwYFNKnrKeDQkFqLxMvTXD83",
                "oid": "EaM4cO2c6AIMWBDwOnQSSfc95qHoF1_8OesVSrK2-O1RLWTEq3QB0cnj46WIa_KIDNq-wpwxRmNQu3-5"
            }
       }
  ],
  "aad": "itXVSOmyKyNvjHYSDiCpvxB0HqlFfy5YJtdQftWqjEk=",
  "iv": "WsiI5jXbXKYUJjwF0H3ZVl02meBL26bL",
  "tag": "jMmIt9MsYRtboMwA4yrf2g==",
  "ciphertext": "VbnjyXkgej1Ix1gjxkVs0ncGWauOxyt5QzY="
}

[TelegramSam]

Continue issue here: https://github.com/decentralized-identity/didcomm-messaging/issues/19