search

hyperledger / aries-rfcs

Hyperledger Aries is infrastructure for blockchain-rooted, peer-to-peer interactions
https://hyperledger.github.io/aries-rfcs/
Apache License 2.0
324 stars 218 forks source link

RFC 0019: Updating to JWE compliant data model #133

Closed kdenhartog closed 4 years ago

kdenhartog commented 5 years ago

Recently there's been discussion to update this work to be fully JWE compliant. One of the proposed solutions that's come up is to generate the following format:

{
    "protected": base64url({
        "typ": "prs.hyperledger.aries-auth-message",
        "alg": "ECDH+XC20PKW",
        "enc":"XC20P"
    }),
    "recipients": [
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base58(Recipient vk)",
                "iv": "base64url(CEK encryption IV)",
                "pk": "compactJWE(Sender PK)"
            }
        },
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base58(Recipient vk)",
                "iv": "base64url(CEK encryption IV)",
                "pk": "compactJWE(Sender PK)"
            }
        }
    ],
    "aad": "base64url(sha256(concat('.',sort([recipients[0].kid, recipients[n].kid]))))",
    "iv": "base64url(content encryption IV)",
    "ciphertext": "base64url(ciphertext)",
    "tag": "base64url(AEAD Authentication Tag)"
}

Exploded example of compactJWE(Sender PK):
{
    "protected": base64url({
        "iv": "base64url(CEK encryption IV)",
        "epk": "Ephemeral JWK",
        "typ": "jose",
        "cty": "jwk+json",
        "alg": "ECDH-ES+XC20PKW",
        "enc":"XC20P"
    }),
    "encrypted_key": "base64url(encrypted CEK)",
    "iv": "base64url(content encryption IV)",
    "ciphertext": "base64url(Encrypted Sender vk as JWK)",
    "tag": "base64url(AEAD Authentication Tag)"
}

Here's a provided example of what this would look like:

{
    "protected": "eyJ0eXAiOiJwcnMuaHlwZXJsZWRnZXIuYXJpZXMtYXV0aC1tZXNzYWdlIiwiYWxnIjoiRUNESCtYQzIwUEtXIiwiZW5jIjoiWEMyMFAifQ",
    "recipients": [
        {
            "encrypted_key": "whpkJkvHRP0XX-EqxUOHhHIfuW8i5EMuR3Kxlg5NNIU",
            "header": {
                "kid": "5jMonJACEPcLfqVaz8jpqBLXHHKYgCE71XYBmFXhjZVX",
                "iv": "tjGLK6uChZatAyACFzGmFR4V9othKN8S",
                "tag": "ma9pIjkQuzaqvq_5y5vUlQ",
                "pk": "eyJpdiI6IldoVGptNi1DX2xiTlQ4Q2RzN2dfNjdMZzZKSEF3NU5BIiwiZXBrIjp7Imt0eSI6Ik9LUCIsImNydiI6IlgyNTUxOSIsIngiOiJnNjRfblJSSFQyYk1JX1hjT1dHRTdJOGdQcU1VWTF4aUNub2J0LVhDUkNZIn0sInR5cCI6Impvc2UiLCJjdHkiOiJqd2sranNvbiIsImFsZyI6IkVDREgtRVMrWEMyMFBLVyIsImVuYyI6IlhDMjBQIn0.4zUt5tOOlcQWskJqxfMi0tNsfUCAzb5_PDfPqQ1h0Vw.xYkeEXV1_cSYFEd6UBMIfl8MWQfHaDex.XSNKTRXye5-iSXQ-aS_vQVZNEgFE6iA9X_KgSRMzihQBMoI1j4WM3o-9dMT9TeSyMvdq3gXt1NpvLdZHpJplahhk3mxMZL-vawm5Prtf.H7a5N-dggwdesjHyJCl06w"
            }
        },
        {
            "encrypted_key": "dDHydlp_wlGt_zwR-yUvESx9fXuO-GRJFGtaw2u6CEw",
            "header": {
                "kid": "TfVVqzPT1FQHdq1CUDe9XYcg6Wu2QMusWKhGBXEZsosg",
                "iv": "7SFlGTxQ4Q2l02D9HRNdFeYQnwntyctb",
                "tag": "9-O6djpNAizix-ZnjAx-Fg",
                "pk": "eyJpdiI6IkV6UjBFaVRLazJCT19oc05qOVRxeU9PVmVLRFFPYVp1IiwiZXBrIjp7Imt0eSI6Ik9LUCIsImNydiI6IlgyNTUxOSIsIngiOiJoU1g1NGt5ZTdsd0pBdjlMaUplTmh4eFhaV1N0M3hLSDBXUmh6T1NOb1c0In0sInR5cCI6Impvc2UiLCJjdHkiOiJqd2sranNvbiIsImFsZyI6IkVDREgtRVMrWEMyMFBLVyIsImVuYyI6IlhDMjBQIn0.qKmU5xO8Z1ZtRBWEjEMixb5VZG0mrY0LnjUGjLqktwg.EG-VOZSC2vLdoO5l2_A37IYvdXCckLZp.D4kgD6bYL1YfXyApk5ESKE2sc8TUiO-QGBtY-M5hcV_F88JPZdsi53Qofxk02ZxPHJZK-abDy45pIMH6-KUMDfE6WKhW3nPQhydPYutv.0SO4VjM8sDH-wGHcEpinTg"
            }
        }
    ],
    "aad": "OGY5ZDIxMDE3YTQ4MTc4YWE5MTk0MWQyOGJmYjQ1ZmZmMTYzYTE3ZjUxYjc4YjA3YTlmY2FlMmMwOTFlMjBhZg",
    "ciphertext": "x1lnQq_pZLgU2ZC4",
    "tag": "2JgOe9SRjJXddT9TyIjqrg",
    "iv": "fDGEXswlWXOBx6FxPC_u6qIuhADnOrW1"
}

I'm curious what other's opinions are on this approach. I personally am not a fan of the bloat the compact JWE adds in order to encrypt the sender's public key. However, I've not found another approach yet that seems satisfactory. If anyone has some suggestions it would be appreciated.

kdenhartog commented 5 years ago

Please be aware, this issue is also posted on the DIDComm-js library in DIF. I'll facilitate communication across the groups if you don't want to track both conversations.

troyronda commented 5 years ago

I would like to see Aries using a JWE compliant data model.

I am in favor of the proposal.

baha-ai commented 5 years ago

the above example is for Authrcrypt, below is the Anoncrypt example: 

{
    "protected": base64url({
        "typ": "prs.hyperledger.aries-anon-message",
        "alg": "ECDH-ES+XC20PKW",
        "enc":"XC20P"
    }),
    "recipients": [
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base58(Recipient vk)",
                "iv": "base64url(CEK encryption IV)",
                "epk": "Ephemeral JWK",
            }
        },
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base58(Recipient vk)",
                "iv": "base64url(CEK encryption IV)",
                "epk": "Ephemeral JWK",
            }
        }
    ],
    "aad": "base64url(sha256(concat('.',sort([recipients[0].kid, recipients[n].kid]))))",
    "iv": "base64url(content encryption IV)",
    "ciphertext": "base64url(ciphertext)",
    "tag": "base64url(AEAD Authentication Tag)"
}
llorllale commented 5 years ago

It's worth pointing to the internet draft that originally defined the JOSE AEAD algorithm for chacha20: https://tools.ietf.org/html/draft-amringer-jose-chacha-00

gamringer commented 5 years ago

I have just updated the draft to include ECDH-ES bits:

https://tools.ietf.org/html/draft-amringer-jose-chacha-01

kdenhartog commented 5 years ago

@gamringer I'm not finding documentation or an IANA registry of the pk field. Do you have details about it?

gamringer commented 5 years ago

I'm writing up the I-D for ECDH for JOSE separately from the Chacha for JOSE I-D. I'm writing it up to have the pk field be the Sender's JWK, or JWE-envelopped JWK, as was the case in my example.

kdenhartog commented 5 years ago

Ok, since we're registering the pk field is it possible that we can try and find a more compact form for this? With ed25519 and secp256k1 keys I'm not too concerned about size (it's not great but at least manageable), but with a 3072 bit RSA key, this is going to going bloat the message to the point where it takes a second or 2 to load the messages and could make it not possible to embed in QR codes.

gamringer commented 5 years ago

Then we can register both opk, and oid to serve are mirror to recipient versions where opk is to jwk what oid is to kid. That way, prs.hyperledger.aries-auth-message can just use oid to hold a crypto_box_seal of the sender's key identifier, since for format for oid (as for kid) is application dependant.

kdenhartog commented 5 years ago

That seems like a good solution that will allow us to work with multiple types then.

baha-ai commented 5 years ago

@gamringer @kdenhartog I pushed an initial PR to our go-jose fork to support the above format using Go: unit tests are passing: https://github.com/trustbloc/go-jose/pull/1 your input is appreciated.

baha-ai commented 5 years ago

That seems like a good solution that will allow us to work with multiple types then.

@gamringer @kdenhartog can we have an updated JWE format with opk and oid examples? I wanna be clear on what fields our JWE will be using to reference these key IDs.

gamringer commented 5 years ago

I finished reimplementing my proposal in PHP: https://github.com/gamringer/php-authcrypt

This is the output of 1-crypt.php:

Sender SK   : 6tsNPgZAg-NWM3s4S0VOOWM2yrcOfwsCrN0JGFEWaWw
Sender PK   : QbvqozxGQ3U8FDLmlKOx8Hd5GiozMRO2pwrevZ5ZFTM
Recipient SK: yOTIYzADtNrszMN0Eq-BE22Rb8v_3ZoybpcxJ-oHhp4
Recipient PK: -u0zk9iY_ZS2wP2z4zuLjR7_kz_kxVU0anRz8_A66T0
Payload     : SGVsbG8gV29ybGQh
--------------------------------
{"protected":"eyJ0eXAiOiJwcnMuaHlwZXJsZWRnZXIuYXJpZXMtYXV0aC1tZXNzYWdlIiwiYWxnIjoiRUNESC1TUytYQzIwUEtXIiwiZW5jIjoiWEMyMFAifQ","recipients":[{"encrypted_key":"y5yrOJUGSZHeJvVjgo39s7_TNRJsCZB46Q_MvNfa70Y","header":{"apu":"bcyJRBCQSQG-h7L7qGge9cMsdXkmMcLNsXvoWq1X1uJzOFyHK-Wg9shZTSQL8XmeW3BzzXvoIWosKw0vfCjtng","iv":"O80MuQSYTjjmNajjTjTlH6etFPpDR7pM","tag":"Y92d_V-qKHA1aedRCjvdFQ","kid":"HtWhz6QevaQF39Gv3Hvf7K6xo2FTViAkY22rhZpQrdWc","oid":"6dhhFIHHi9UN-N9IJr0XTRiijKvF0aSwxg7MTMAH_AuNc8THkblgbEGvW1Y8r2R02iuJbGj_jkMd_d899A6God8Dyw0B-TR9Lh7N6ho3-eEic-XKM5MGgLR2gg"}}],"aad":"FeI0LXy7m0-orE0VwiQU-2RjQyYMsnIvSEzpduiB7sY","iv":"ofkD-Q0peUhkJzrDiM2mlC-O5HgyTBmZ","tag":"osug2xz38IjB_oqubJXkcw","ciphertext":"vIPC1fCfxIMkxDsu"}
--------------------------------
{
    "protected": "eyJ0eXAiOiJwcnMuaHlwZXJsZWRnZXIuYXJpZXMtYXV0aC1tZXNzYWdlIiwiYWxnIjoiRUNESC1TUytYQzIwUEtXIiwiZW5jIjoiWEMyMFAifQ",
    "recipients": [
        {
            "encrypted_key": "Y6eYXyrOWj67oHh4hla_MS024oPtgWeM4LOPqwyrXTM",
            "header": {
                "apu": "MMapegFCsTTrygbuC80X0NeHjrtJ7Fh5d9CIl6pq4HVgYDAtjIS7dyQKXO-Vgan8ho33ZglRJCfW4Wx2pH3cNg",
                "iv": "eBuzpjLTU16gmJvZKV3JShzvibJM6h7_",
                "tag": "PKg4RLQ5hikKQ2Vq2SCqGg",
                "kid": "HtWhz6QevaQF39Gv3Hvf7K6xo2FTViAkY22rhZpQrdWc",
                "oid": "cOvRYUooq-y1TDjK3Lt3wCA3H-w9E6PJXNPDdIPLDDZlpE7QJvJuLIwJzSPqDhRvQUMaOYrXyLGAgdriGpKbKjWcLtQapjExq8sesL5bax68J46vv-2-GuDVbQ"
            }
        }
    ],
    "aad": "FeI0LXy7m0-orE0VwiQU-2RjQyYMsnIvSEzpduiB7sY",
    "iv": "9AL-EASXKfuonBKKxsPHSccrX2zy7j2l",
    "tag": "IG6L99-sFnq3Cfz29Z-jDg",
    "ciphertext": "IX7EQSrqhxL61YjE"
}
baha-ai commented 5 years ago

I believe we should update the format to meet the standards and reference them in the docs.

@gamringer, I gathered these links from the discussion we had such as:

https://tools.ietf.org/html/rfc7518#section-4.6.2

More specifically about the alternative format for apu with a random 64 byte value and no apk:

   a manner similar to "Diffie-Hellman Key Agreement Method" [RFC2631]:
   in that case, the "apu" parameter MAY either be omitted or represent
   a random 512-bit value (analogous to PartyAInfo in Ephemeral-Static
   mode in RFC 2631) and the "apv" parameter SHOULD NOT be present.

   See Appendix C for an example key agreement computation using this
   method.

we should probably update the format to reflect these changes

baha-ai commented 5 years ago

@kdenhartog @gamringer, I just pushed the Go implementation of the JWE encryption (decryption will be coming up soon). Here is an example format: Encryption with XC20P: 

{
  "protected": {
  "alg": "ECDH-SS+XC20PKW",
  "enc": "XC20P",
  "typ": "prs.hyperledger.aries-auth-message"
  },
  "recipients": [
       {
          "encrypted_key": "C1ZgZxgIKne86FlGuct0L2y2UENtHyOS78KvoVgBAaM=",
          "header": {
                "apu": "dpQnrrNMi8CHC1sRLxcKdzHUfYqM9hMI4P8VpLMArVYdJQlmnIzF6niGjzTCNF6UBobaTmm6msTlEK2chntsOA==",
                "iv": "Po7Wi7CKWHeCof450oyrFB9A_x8D8LQa",
                "tag": "QkkS6QqQLaWRJDtEK0htKQ==",
                "kid": "ETu2KkCZUawEHq99z2RubyCsnVyaLhzMiYTHMu3uVncp",
                "oid": "PBqOfOOkMM3hDUxSunySlld6-15qdbMdWajolmmKVYs0bog785bnqckvoPYDkqoMXwVPtr-qPQgIpg9A"
            }
       },
       {
          "encrypted_key": "pGL8MBT1WUXjfXuwSl2enj5N1BLtmeEZ1wAZAzUoH60=",
          "header": {
                "apu": "iSa2FM-_nOzxG4IjX5MOUv1uvEFs-nA-fYlrLry66VVrNJxakXclPC7UnJ_RtqI_cwlEgJ_GaVHcNv84NA8VjQ==",
                "iv": "xSEHpsn1iuy4x3KTfeHdcdBRziqCvImG",
                "tag": "vsU_H6K5m4_jVFWNxxV--A==",
                "kid": "BkcdRt9AChfFXGvKU3ecjzyNCzoP4EBVvm5dtgvqv5UX",
                "oid": "u1PbWv2wIIV4BkOibItT4DzkzVb6d75hLhS01Ha1CzapUX5xjBlcyjwrl6nIOrARBvoJyH0FXZdkxec7"
            }
       },
       {
          "encrypted_key": "PsFDcK7-VHzH-E-KDC-Cy1NHZYKepv1X7zolbuRbdbw=",
          "header": {
                "apu": "PAYUGCNWRKGq82HvOxyMVE6VI13mYlvowK5X3NqZsFiwqpTTwXgNTZA5Tnxch8vsBZAzeEpmEvI3M9YpJflnrA==",
                "iv": "s_FfFvulVhFKwCoalqp9qESsEtWnDgZ9",
                "tag": "7IIQuM9Hs2Zqsj9Vga2nAQ==",
                "kid": "EikHy4Hmbw5zGUMPGYDoAwYFNKnrKeDQkFqLxMvTXD83",
                "oid": "EaM4cO2c6AIMWBDwOnQSSfc95qHoF1_8OesVSrK2-O1RLWTEq3QB0cnj46WIa_KIDNq-wpwxRmNQu3-5"
            }
       }
  ],
  "aad": "itXVSOmyKyNvjHYSDiCpvxB0HqlFfy5YJtdQftWqjEk=",
  "iv": "WsiI5jXbXKYUJjwF0H3ZVl02meBL26bL",
  "tag": "jMmIt9MsYRtboMwA4yrf2g==",
  "ciphertext": "VbnjyXkgej1Ix1gjxkVs0ncGWauOxyt5QzY="
}

Go PR: https://github.com/hyperledger/aries-framework-go/pull/107

as mentioned in my previous post, we need to update the format a little and confirm if we're using oid or sid (or any other fields)

kdenhartog commented 5 years ago

Update on this. I've been looking at this recently and trying to weigh all factors that would get us good interop between all parties in DID community. Since this change would create a breaking change, I want to make sure that what ever we move to accounts for all of our needs for quite some time.

For example, one thing that I didn't account for that I'd like to in the future is the ability to add in forward secrecy when we perform this breaking change.

Since I know this work is blocking @Baha-sk my recommendation is to implement the currently non-compliant JWE for now and we'll say that it is version 1. This will give us time to figure out what we want and to conduct a larger discussion as to what we should do going forward. Does that feel like a reasonable approach?

troyronda commented 5 years ago

Thanks @kdenhartog for taking the time to look at this issue, and your continuing efforts.

We would like to use a compliant JWE. We have already started implementing this updated approach.

I am concerned about:

With these concerns, I cannot say that deferring the compliant version is the correct path. It would be much better to become compliant as soon as possible (as the v1), and then iterate from there as needed.

baha-ai commented 5 years ago

Thanks for the update @kdenhartog

I do agree with @troyronda

There has been some effort done to make it JWE compliant, it makes sense to finish off the work and be JWA compliant. The sooner we deliver this the better (before v1) to avoid backward compatibility issues between agents.

kdenhartog commented 5 years ago

After our discussion on the call today about key identifiers, I was convinced that the explicitness of JWKs will be more beneficial then the conciseness of the libsodium option. @gamringer do you still have to data model that you originally described like this? Would we need to change anything to make that the standard implementation?

gamringer commented 5 years ago

@kdenhartog I did not reimplement that version of my proposal in a poc after my workstation failure. I can take a couple of hours to do it.

baha-ai commented 5 years ago

After our discussion on the call today about key identifiers, I was convinced that the explicitness of JWKs will be more beneficial then the conciseness of the libsodium option. @gamringer do you still have to data model that you originally described like this? Would we need to change anything to make that the standard implementation?

@kdenhartog is this confirmed to be the final change about the JWE encryption? Because I believe it's simpler to just box.Open the base58 decoded OID with the recipient's private key than having to JWE unmarshal spk then extract the key from the jwk (it's less work).

Mind you, none of this is part of the JWE standard yet, so we still have a chance at making it simpler.

what do you think?

llorllale commented 5 years ago

@kdenhartog can you elaborate on why we should not include a standard sender attribute in DIDComm plaintext messages?

troyronda commented 5 years ago

@Baha-sk I believe we should make our design choices based on what is most consistent with the existing JOSE specs (regardless of extra work). I would assume wrapping a key in a JWK would be more consistent...

baha-ai commented 5 years ago

I guess we want to stick to JWE/JWK format as much as possible @kdenhartog , this is why we want the sender's key to be in JWK format?

llorllale commented 5 years ago

@kdenhartog reminder: the discussion on the last call revolved around two points: 1) how to encode the inline key such that its format can be interpreted, and 2) how to identify the Sender of the DIDComm msg.

It is understood that JWKs solve problem 1. Are you proposing to also solve problem 2 with them?

kdenhartog commented 5 years ago

@kdenhartog reminder: the discussion on the last call revolved around two points: 1) how to encode the inline key such that its format can be interpreted, and 2) how to identify the Sender of the DIDComm msg.

It is understood that JWKs solve problem 1. Are you proposing to also solve problem 2 with them?

Sorry, I've been away at RWoT, but was able to do some investigation into this. I do believe that JWKs would allow us to solve both problems. The JWK format provides a kid field which can list the DID URI, while also containing all the necessary cryptographic material in the other parts of the JWK format.

@selfissued Would you mind taking a look at this discussion and validating the direction we're going as well? This is the discussion I mentioned to you at RWoT.

kdenhartog commented 5 years ago

@gamringer I searched through the rocketchat and found your old messages about this. I'm going to copy it here so it's easier to find.

Authcrypt with Non-repudiable:

{
    "protected": base64url({
        "typ": "prs.hyperledger.aries-anon-message",
        "alg": "ECDH-ES",
        "enc":"XC20P"
    }),
    "recipients": [
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base58(Recipient vk)",
                "epk": "Ephemeral JWK",
            }
        },
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base58(Recipient vk)",
                "epk": "Ephemeral JWK",
            }
        }
    ],
    "aad": "base64url(sha256(concat('.',sort([recipients[0].kid, recipients[n].kid]))))",
    "iv": "base64url(content encryption IV)",
    "ciphertext": base64url(XC20P(CompactJWS({
        "protected": base64url({
            "alg": "EdDSA",
            "kid": base58("Sender vk")
        }),
        "payload": base64url("Wire message"),
        "signature": base64url("signature")
    }))),
    "tag": "base64url(AEAD Authentication Tag)"
}

exploded JWE for encrypted JWK so that we get the sender anonymity property: Exploded example of compactJWE(Sender PK):

{
    "protected": base64url({
        "iv": "base64url(CEK encryption IV)",
        "epk": "Ephemeral JWK",
        "typ": "jose",
        "cty": "jwk+json",
        "alg": "ECDH-ES+XC20PKW",
        "enc":"XC20P"
    }),
    "encrypted_key": "base64url(encrypted CEK)",
    "iv": "base64url(content encryption IV)",
    "ciphertext": "base64url(Encrypted Sender vk as JWK)",
    "tag": "base64url(AEAD Authentication Tag)"
}

Anoncrypt option

{
    "protected": base64url({
        "typ": "prs.hyperledger.aries-anon-message",
        "alg": "ECDH-ES+XC20PKW",
        "enc":"XC20P"
    }),
    "recipients": [
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base64url(Recipient vk)",
                "iv": "base64url(CEK encryption IV)",
                "epk": "Ephemeral JWK",
            }
        },
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "kid": "base64url(Recipient vk)",
                "iv": "base64url(CEK encryption IV)",
                "epk": "Ephemeral JWK",
            }
        }
    ],
    "aad": "base64url(sha256(concat('.',sort([recipients[0].kid, recipients[n].kid]))))",
    "iv": "base64url(content encryption IV)",
    "ciphertext": "base64url(ciphertext)",
    "tag": "base64url(AEAD Authentication Tag)"
}

example

{
    "protected": "eyJ0eXAiOiJwcnMuaHlwZXJsZWRnZXIuYXJpZXMtYXV0aC1tZXNzYWdlIiwiYWxnIjoiRUNESCtYQzIwUEtXIiwiZW5jIjoiWEMyMFAifQ",
    "recipients": [
        {
            "encrypted_key": "whpkJkvHRP0XX-EqxUOHhHIfuW8i5EMuR3Kxlg5NNIU",
            "header": {
                "kid": "5jMonJACEPcLfqVaz8jpqBLXHHKYgCE71XYBmFXhjZVX",
                "iv": "tjGLK6uChZatAyACFzGmFR4V9othKN8S",
                "tag": "ma9pIjkQuzaqvq_5y5vUlQ",
                "pk": "eyJpdiI6IldoVGptNi1DX2xiTlQ4Q2RzN2dfNjdMZzZKSEF3NU5BIiwiZXBrIjp7Imt0eSI6Ik9LUCIsImNydiI6IlgyNTUxOSIsIngiOiJnNjRfblJSSFQyYk1JX1hjT1dHRTdJOGdQcU1VWTF4aUNub2J0LVhDUkNZIn0sInR5cCI6Impvc2UiLCJjdHkiOiJqd2sranNvbiIsImFsZyI6IkVDREgtRVMrWEMyMFBLVyIsImVuYyI6IlhDMjBQIn0.4zUt5tOOlcQWskJqxfMi0tNsfUCAzb5_PDfPqQ1h0Vw.xYkeEXV1_cSYFEd6UBMIfl8MWQfHaDex.XSNKTRXye5-iSXQ-aS_vQVZNEgFE6iA9X_KgSRMzihQBMoI1j4WM3o-9dMT9TeSyMvdq3gXt1NpvLdZHpJplahhk3mxMZL-vawm5Prtf.H7a5N-dggwdesjHyJCl06w"
            }
        },
        {
            "encrypted_key": "dDHydlp_wlGt_zwR-yUvESx9fXuO-GRJFGtaw2u6CEw",
            "header": {
                "kid": "TfVVqzPT1FQHdq1CUDe9XYcg6Wu2QMusWKhGBXEZsosg",
                "iv": "7SFlGTxQ4Q2l02D9HRNdFeYQnwntyctb",
                "tag": "9-O6djpNAizix-ZnjAx-Fg",
                "pk": "eyJpdiI6IkV6UjBFaVRLazJCT19oc05qOVRxeU9PVmVLRFFPYVp1IiwiZXBrIjp7Imt0eSI6Ik9LUCIsImNydiI6IlgyNTUxOSIsIngiOiJoU1g1NGt5ZTdsd0pBdjlMaUplTmh4eFhaV1N0M3hLSDBXUmh6T1NOb1c0In0sInR5cCI6Impvc2UiLCJjdHkiOiJqd2sranNvbiIsImFsZyI6IkVDREgtRVMrWEMyMFBLVyIsImVuYyI6IlhDMjBQIn0.qKmU5xO8Z1ZtRBWEjEMixb5VZG0mrY0LnjUGjLqktwg.EG-VOZSC2vLdoO5l2_A37IYvdXCckLZp.D4kgD6bYL1YfXyApk5ESKE2sc8TUiO-QGBtY-M5hcV_F88JPZdsi53Qofxk02ZxPHJZK-abDy45pIMH6-KUMDfE6WKhW3nPQhydPYutv.0SO4VjM8sDH-wGHcEpinTg"
            }
        }
    ],
    "aad": "OGY5ZDIxMDE3YTQ4MTc4YWE5MTk0MWQyOGJmYjQ1ZmZmMTYzYTE3ZjUxYjc4YjA3YTlmY2FlMmMwOTFlMjBhZg",
    "ciphertext": "x1lnQq_pZLgU2ZC4",
    "tag": "2JgOe9SRjJXddT9TyIjqrg",
    "iv": "fDGEXswlWXOBx6FxPC_u6qIuhADnOrW1"
}

It's worth noting that as I read through these, they haven't been updated to account for our discussion around moving to JWKs and defining the format such that they include the DID-url with a key fragment and defining how it's encrypted.

dhh1128 commented 4 years ago

I hadn't been very aware of this conversation (although I knew it was happening in theory). I wanted to articulate two requirements that I consider important:

  1. Preserve the option to operate in repudiable mode. (This is an absolute dealbreaker for me; if we lose this, I can't use the solution, period. I believe from scanning the thread and from personal conversations with Kyle that this requirement is well understood and we are tracking it to be satisfied by the final solution, so me articulating it may be unnecessary--but I'm just checking.)

  2. Solve the problem of runaway bloat caused by repeated re-encryption. (This is NOT an absolute dealbreaker for me, but is highly desirable. We proposed a few months back to change the encryption envelope to address this, but the proposal was deferred because we didn't think it was worth it to modify the envelope format solely to add this feature. Now that we're considering a change to the format for different reasons, I strongly prefer that we tackle this problem at the same time, and not defer it again.)

Am I correct that requirement #1 is taken care of? And where are we at with requirement #2?

dhh1128 commented 4 years ago

A clarification: I see Kyle's comment in the stream about how to get sender anonymity. What I'm asking is whether "sender anonymity" is truly the same thing as "repudiable origin". Under some definitions, these might be synonyms; under others, maybe not. To me, "repudiable origin" means that nobody other than the sender and the receiver can know with certainty who the sender is--whereas sender anonymity could mean the sender is not known even to the receiver.

dhh1128 commented 4 years ago

I have another requirement to posit:

  1. Decryption cannot depend on DID resolution.

We can put a key ID in a kid field, and the value we put there can be a DID-based key reference. However, the decryption itself can't require a dereference, because we end up with a timing problem (the key value at the time of dereference isn't guaranteed to be the key value at the time of encryption). So we must be decrypting with something other than kid. This means kid is a hint or advisory field, useful for checking whether the decryption key has the DID context we're hoping for--but is never primary input to decryption.

Is our solution doing that?

baha-ai commented 4 years ago

PHP and Go implementations are now complete using a JWK structure for the sender verification key embedded in a full JWE format, here is an example of Authcrypt message in pseudo code:

{
    "protected": base64url({
        "typ": "prs.hyperledger.aries-anon-message",
        "alg": "ECDH-SS+XC20PKW",
        "enc":"XC20P"
    }),
    "recipients": [
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "apu": "base64url(random_bytes(64))",
                "nonce": "base64url(random_bytes(nonce_size))",
                "tag": "base64url(encrypted CEK tag)",
                "kid": "base58_encode(Recipient vk)",
                "spk": "EphemeralJWK(Sender vk, Recipient vk)",
            }
        },
        {
            "encrypted_key": "base64url(encrypted CEK)",
            "header": {
                "apu": "base64url(random_bytes(64))",
                "nonce": "base64url(random_bytes(nonce_size))",
                "tag": "base64url(encrypted CEK tag)",
                "kid": "base58_encode(Recipient vk)",
                "spk": "EphemeralJWK(Sender vk, Recipient vk)",
            }
        }
    ],
    "aad": "base64url(sha256(concat('.',sort([base58(recipients[0] vk), ..., base58(recipients[n] vk ])))))",
    "iv": "base64url(content encryption IV)",
    "ciphertext": "base64url(XC20P(DidExchange payload, base64Url(json($protected)+'.'+$aad), content encryption IV, CEK))"
    "tag": "base64url(AEAD Authentication Tag)"
}

Where EphemeralJWK(Sender vk, Recipient vk) outputs a compact JWE format defined as follows:

base64Url(Headers).base64Url(encryptedKEK).base64Url(random_bytes(nonce_size)).base64Url(encryptedJWK cipher).base64Url(encryptedJWK tag)

where:

XC20P is the xchacha20poly1305 cipher Seal()/Open() calls in Go or sodium_crypto_aead_xchacha20poly1305_ietf_encrypt() / sodium_crypto_aead_xchacha20poly1305_ietf_encrypt() in PHP

Example:

Sender SK   : W_WR77SQUhHikPHXn-CP5SynZ8ebMIh7SGe5GTEnbcw
Sender PK   : sapGsej_in9ISk06-2l9dimNHewl5f5-U9FKAnMcH2U
Recipient SK: Y92S7bhd8dJR-9X5yVrLUVlEH0vdNfd52No7aMftW7A
Recipient PK: a3iqc1ORk_y_HvfZtncBPxgWaVM3NA2cWLyf0OG9qQc
Payload     : SGVsbG8gV29ybGQh. // base64Url("Hello World!")
Headers: 
{"typ":"prs.hyperledger.aries-auth-message","alg":"ECDH-SS+XC20PKW","enc":"XC20P"}
Envelope:
{
    "protected": "eyJ0eXAiOiJwcnMuaHlwZXJsZWRnZXIuYXJpZXMtYXV0aC1tZXNzYWdlIiwiYWxnIjoiRUNESC1TUytYQzIwUEtXIiwiZW5jIjoiWEMyMFAifQ",
    "recipients": [
        {
            "encrypted_key": "rPy69f85-4oqdSF2pmOpxw6FXE9ID--XelyKndkGJ7k",
            "header": {
                "apu": "TlhMaHseS0urFH3qfwFhBluPBZekR6Ro_haJJvpxVs64bkXP5jH3-eGyirmgMkxr9FgAx6Zieau2EsseQs6qqg",
                "iv": "kWLiKhsER0FGgSahLjL4HAe5W8d98Ol_",
                "tag": "-d8xg42QPabb_oOkHxDcuw",
                "kid": "8EXLnYec1Lh2w92WSyo5KWaBwD3QaEKjzahFCp4QaTH8",
                "spk": "eyJ0eXAiOiJqb3NlIiwiY3R5IjoiandrK2pzb24iLCJhbGciOiJFQ0RILUVTK1hDMjBQS1ciLCJlbmMiOiJYQzIwUCIsImVwayI6eyJrdHkiOiJPS1AiLCJjcnYiOiJYMjU1MTkiLCJ4IjoiLVFOOFpLR2VrcTQyS09DcnVSRWdVZDk0YUFCd2l6alpHUzlmSWR1YXlVUSJ9LCJpdiI6IlgtaWx1SHFHOEhwbDFTMHNSQVR3MFlWYk1ZRDlVdHlIIiwidGFnIjoiRXpEbTl0cjdNOS1DOVRFNURseTc0dyJ9.6wzR556W_NtyPq7o55Giou0MJTGSfT1eYZhV2hv_GlE.3mB89GEslNpzkCo_5BrG36XlYE2xXeMY.QXwOr3AmH_Xinoe6-4fnhoUYrm3f9py8vAdTrua3BTutsYnZbr7OQIbSQqy4lO7TwsZ8KdJtfijFlbhdlrFUbPL3RdmqXcx3UGgw2FHg.vRe7YRKtYnAnDin9qwlPwA"
            }
        }
    ],
    "aad": "hgMzrJYQWNginfZSRF2dPEm01XkyEetXWJj_QDb5hmA",
    "iv": "g0Z7v-Fl02MgIZPdsEhDUp_Z4ZGvP0yk",
    "tag": "ioH7xaCdDSwzRG4Hhp1VHw",
    "ciphertext": "wOb0codq4GydjtqB"
}

PHP implementation: https://github.com/gamringer/php-authcrypt aries-framework-go Agent implementation: https://github.com/hyperledger/aries-framework-go/tree/v0.1.0/pkg/didcomm/packer/jwe/authcrypt

dhh1128 commented 4 years ago

@Baha-sk This shows that kid contains a recipient's verkey. That's fine with me (it satisfies requirement 3 that I articulated here)--but I thought others were advocating that kid should contain a DID-based key reference instead. See https://github.com/hyperledger/aries-rfcs/issues/104#issuecomment-525396519

baha-ai commented 4 years ago

@dhh1128 for now, the PHP and Go implementations are using recipient verkey as kid value (I've changed my previous post above to showcase the base58 encoding of these keys).. the purpose of this code is to run within the wallet (as part of Pack()/Unpack()).

We probably can switch kid to use DID-based key references but I guess the code needs to be refactored to extract Pack/Unpack out of the wallet and call crypto code with key materials inside the wallet.. it will need further refactoring if we opt to go this route.

dhh1128 commented 4 years ago

Right. I like that approach. It is what I was recommending. However, I got the impression that some in the community were advocating the opposite, so I was just noting that there may be pushback.

baha-ai commented 4 years ago

I don't see any harm in sharing public keys in the envelope

llorllale commented 4 years ago

@Baha-sk

base64Url(Headers).base64Url(encryptedKEK).base64Url(random_bytes(nonce_size)).base64Url(encryptedJWK cipher).base64Url(encryptedJWK tag)

Encrypted JWKs are JWEs with the entire JWK as the plaintext - see https://tools.ietf.org/html/rfc7517#appendix-C, not just the KEK.

baha-ai commented 4 years ago

@llorllale the value of the recipient header's spk field is the JWE in compact format. The second field in the JWE is the encryption key (encryptedKEK) which is used to encrypt the CEK for the corresponding recipient.

see the final section (9) of the same appendix-C

llorllale commented 4 years ago

Right, my bad - I mistook the ordering of the segments.

llorllale commented 4 years ago

@dhh1128

We can put a key ID in a kid field, and the value we put there can be a DID-based key reference. > However, the decryption itself can't require a dereference, because we end up with a timing problem (the key value at the time of dereference isn't guaranteed to be the key value at the time of encryption). So we must be decrypting with something other than kid. This means kid is a hint or advisory field, useful for checking whether the decryption key has the DID context we're hoping for--but is never primary input to decryption.

Is our solution doing that?

We will add the DID-based key reference as a kid in the JWK that encodes the sender's verKey (the spk in Baha's snippets). Not to be confused with recipient.header.kid, which contains the recipient's inlined key.

llorllale commented 4 years ago

@dhh1128 can you elaborate on this point? I'm not sure what you're referring to:

Solve the problem of runaway bloat caused by repeated re-encryption. (This is NOT an absolute dealbreaker for me, but is highly desirable. We proposed a few months back to change the encryption envelope to address this, but the proposal was deferred because we didn't think it was worth it to modify the envelope format solely to add this feature. Now that we're considering a change to the format for different reasons, I strongly prefer that we tackle this problem at the same time, and not defer it again.)

dhh1128 commented 4 years ago

@llorllale: This sounds like an incomplete solution. Let me posit a scenario and then you can tell me if I'm understanding wrong.

Alice sends a message to Bob. When she sends the message on Monday, she uses one of her keys that is known as did:example:alicedid#1, with public key value ABCXYZ, and she encrypts FOR Bob's public key that is known as did:example:bobdid#3, with public key value DEFPQR. Thus, if I understand you right, the string did:example:alicedid#1 gets put into spk, and the string DEFPQR gets put into recipient.header.kid. So far, do I have this right?

Now, Bob loses his phone in the couch cushions for 3 days, so he doesn't actually open Alice's message until Thursday. But on Wednesday, Alice rotated her key, such that did:example:alicedid#1 now has public key value GHIJKLM.

When Bob opens the message, he can decrypt it immediately, because he sees DEFPQR (a key he knows about) in the recipient.header.kid field. Then, having decrypted the message, he decides to validate the sender. So he takes what he sees in spk, which is the string did:example:alicedid#1, resolves Alice's DID, and learns that it has the value GHIJKLM. When he checks to see if the key used to encrypt the message corresponds to that key, the check fails (Alice encrypted with ABCXYZ). It appears to Bob like Alice is lying, because the DID reference has an indeterminate timeframe relative to the actual encryption.

What I think we need is the actual VALUE, not a REFERENCE, to the sender's public key at the time of message sending. We could also have a reference--but the value, not the reference, is what needs to be used to do primary validation on the sender. Then, as a secondary step, it might be desirable to confirm that the sender's verkey did (at some timeframe in the past that roughly corresponded to send time) have the id property implied by the key reference hint.

dhh1128 commented 4 years ago

@llorllale : The "runaway bloat" problem is pictured here: https://docs.google.com/presentation/d/1zE5YLzU2VUFdRy3lNpv4LOwMy15yJAkZMO_eitfNSOk/edit#slide=id.g4d0fe0a930_0_0. Take a minute to follow the link; I think a picture is worth a thousand words.

But here are some words to go with it: basically, every time you wrap content in an encryption transformation, you might be re-enccrypting content. If a particular DIDComm channel has 5 hops, then that's 1 necessary encryption and 4 unnecessary ones. Not only does this waste CPU cycles, it also increases message sizes--maybe a lot.

What we'd like to do at each new wrapping is encrypt just the metadata from the previous message, but not re-encrypt the already encrypted payload.

baha-ai commented 4 years ago

@dhh1128 I don't think we have a "runaway bloat" issue with our embedded jwk solution above. The encryption from our solution happens more like this:

Encryption 1 (main outer JWE):
    CEK --> enc --> Payload
Encryption 2 (recipient A CEK):
    KEKA --> enc --> CEK
Encryption 3 (sender's JWK as payload for Recipient A represented as an inner compact JWE):
    CEKA --> enc --> Sender JWK
Encryption 4 ( recipient A's sender CEK)
    SenderKEKA --> enc --> CEKA

Encryptions 2, 3 and 4 are repeated for as many recipients as we have on the list. We are not encrypting Payload or CEK over and over for each recipient.

dhh1128 commented 4 years ago

@Baha-sk I'm not saying the bloat is affected by the number of recipients. I'm saying it's affected by the number of times re-encryption happens. What if I take a fully encrypted message (encrypted for, say, 3 recipients--or 1 or 20 or whatever) and encrypted it again, for 2 different recipients, and then encrypt it again for 1 yet different recipient?

I think you are assuming that the encryption function is called once with all recipients, and that's it. I'm assuming an onion routing scenario like this:

Alice wants to send to Bob, via the intermediate routers X, Y, and Z. So the message undergoes the following encryptions:

encrypt( encrypt( encrypt( encrypt(msg, for=[Bobkey1, Bobkey2, Bobkey3, Bobkey4]), for=[Z]), for=[Y]), for=[X])

In this scenario, you can't put all of Bob's keys and X and Y and Z all into the single recipients array, because they are supposed to see different things. X, Y, and Z are only supposed to see an encrypted payload that they forward on to the next hop in the chain. They shouldn't know the eventual recipient or who preceded them in the chain.

baha-ai commented 4 years ago

@dhh1128 I believe this depends on how we handle routing.

If middle agents are only routing the message then these agents should probably not do Pack/Unpack as they will cause an onion routing effect as you described above.

I think we need to update the RFC to mention how routing between agents should handle the payload it's receiving to avoid encrypting the payload over and over for each middle agent.

In my previous post I was referring to a single agent communication who is sending a payload to multiple recipients regardless of routing agents ( I did not know the list of recipients are basically routing agents + the final agent).

I do think each routing agent should simply pass through the envelope as-is so that the end recipient knows the route of the message. Is there an RFC for this?

But that doesn't change the scenario from my previous post, Encryption 1 happens only once, and the 3 other ones are for each recipient.

Whether the sending agent knows the full route or only the end recipient and the next agent in the route, that's up for a separate discussion.

dhh1128 commented 4 years ago

@Baha-sk : The routing behavior of DIDComm is already set, and it already includes onion routing, as spec'ed in several RFCs. See, for example, RFC 0046 and RFC 0094. This is not going to change, and we can't ignore it in our encryption solution, or demand that it change for encryption convenience. There will be multiple layers of re-encryption. We can't only encrypt once, because then every intermediate router sees the final destination and can monitor where traffic is going--a privacy violation. This means that each hop in the journey of a message decrypts the metadata that tells it where to route for the next hop. Also, there is not just one final agent receiving a message. If Alice is the recipient, and she owns a laptop, an iPhone, and a tablet, then 3 agents will be the recipients at the end.

baha-ai commented 4 years ago

@dhh1128 in this case you're right, each hop is responsible of encrypting (and wrapping) the envelope for the following hop in the route. Then we can't do without re-encrypting the envelop at each hop.

Also, there is not just one final agent receiving a message. If Alice is the recipient, and she owns a laptop, an iPhone, and a tablet, then 3 agents will be the recipients at the end.

I believe the sender should/may not be aware of how many agents Alice has. A middle hop in the route is most probably aware of Alice's three agents via say an email account or a cloud service where Alice's devices are registered. This intermediate hop will encrypt and routes the message to all three agents using their corresponding verification keys.

dhh1128 commented 4 years ago

I believe the sender should/may not be aware of how many agents Alice has. A middle hop in the route is most probably aware of Alice's three agents via say an email account or a cloud service where Alice's devices are registered. This intermediate hop will encrypt and routes the message to all three agents using their corresponding verification keys.

If the final router before the ultimate destination is an agent of the final recipient, it may be acceptable for that router to see the message as plaintext and and re-encrypt it. However, generally this is not the case, and we certainly can't build a routing solution that depends on this behavior. The more common pattern, and the only safe default, is to assume that only the final recipients at the edge should see plaintext, and that the sender must therefore multiplex-encrypt for all the receiver's edge keys.

The sender does not know how many agents Alice has. However, the sender does have Alice's DID Doc, and does know how many keys Alice has that are not listed in endpoint definitions as intermediate routing keys. It is that set of keys that the sender must encrypt for in the recipients array.

Moopli commented 4 years ago

@Baha-sk

I believe the sender should/may not be aware of how many agents Alice has. A middle hop in the route is most probably aware of Alice's three agents via say an email account or a cloud service where Alice's devices are registered. This intermediate hop will encrypt and routes the message to all three agents using their corresponding verification keys.

I'm not sure how such an intermediate agent could reroute a message without either:

For clarity: if A sends a message addressed to C, routed through B, with the expectation that B is a rerouting agent that will redirect the message to C1, C2, C3 etc, without A knowing the identities of C_1, C2, C3 etc, then either:

For some uses (eg, a trusted cloud agent that you expect will do processing based on your data and also pass messages along) this is fine, but I don't think this is fine for every use case, though I defer to others to suggest the use cases.

dhh1128 commented 4 years ago

@Moopli I agree with your summary of the issue. This is exactly the analysis that's described in detail in RFC 0046. Mediators are NOT trusted with plaintext. Rather, they receive an envelope that contains an encrypted payload. The only thing the mediators can understand from looking at the envelope is that a routing request is being made, and the next destination in the route is X. Everything else is hidden.