Should we use `PortableDid` concept in our test vectors?

[jiyoontbd]

context: https://github.com/TBD54566975/web5-spec/pull/130#discussion_r1536259629

[KendallWeihe]

There are two things here

1. Should the concept of a portable DID be within the web5 specification?
2. Should we create two distinct groups of test vectors, one which is a 1:1 comprehensive set of what's in the web5 specification and another which has characteristics which are unique to are given web5 implementations (for example, specific function signature design)?

cc @decentralgabe @mistermoe @frankhinek please use this ticket to discuss (as opposed to the PR where this originated)

[KendallWeihe]

My initial take is, no to (1) and yes to (2). Portable DID is an implementation detail of our implementations, but not necessary for interop with outside implementations. And as such, we should introduce two sets of test vectors, one with a comprehensive set of vectors covering the specification and a different for our implementation details. The merit for the latter is, we want a degree of consistency across our design space (primary (and maybe only?) example is function signatures) which is specific to us (TBD) but isn't strictly necessary for interop.

Basically, my take is, portable DIDs are great for development purposes, but not ideal for standards specifications (b/c of comment below).

Open to arguments against!

---

My primary reasoning for (1) is reflected in this comment from @decentralgabe (from here)

The DID working group argued about a similar concept for a while and agreed to not define a serializable representation of private keys as to avoid recommending doing it, since it's a security risk to transmit private keys.

[frankhinek]

It might be good to talk through this live. My current perspective is that the comparison to the decision by the DID working group to not include private keys in the DID data model is not an apples to apples comparison. As a result, it may not follow that we should draw the same conclusion given that the usage and motivation are different.

That being said, the idea of having two sets of test vectors seems reasonable:

1. a set of test vectors that any implementer (incl. outside TBD) could use to validate spec conformance / interoperability
2. a set of test vectors used by TBD SDK implementers to validate that each language-specific SDK takes similar (but NOT exact) approaches to overall design, data structures that surfaced to external callers, etc.

[decentralgabe]

I chatted with @frankhinek about this earlier today and here is my takeaway:

There is value in having a concept of a "Portable DID" that includes a DID Document and associated metadata (e.g. type values for usage with DID DHT, information around when it was last update or resolved, etc.). This is a distinct concern from providing (and promoting) a serializable form of private key material.

My general concern is that, as much as possible, private keys should not be portable. They should stay within a KMS/HSM, secure enclave/element, or other protected storage as much as possible to avoid a wide array of attack vectors such as accidental exposure/leakage, MiTM attacks, lack of tamper resistance (like that provided by HSMs), opens up risk in key mismanagement, vulns related to key loggers and memory scrapers, and probably more too.

There are cases, such as usages of Ed25519 keys where using a HSM isn't practical in most environments and we need workarounds, such as using AWS Secrets Manager which can provide encryption-at-rest of sensitive material, including cryptographic keys. For usage in memory we should follow secure memory patterns, implementations of which can be found in multiple languages. This requirement can be generalized to we should provide and enable secure access to cryptographic keys consistently across SDKs.

I believe we can achieve that requirement without including private keys in our portable DID construct, but instead providing a common interface like KeyAccess that is able to connect to secure key storage whatever that mechanism may be. This is the approach that the widely used Tink Cryptography library from Google follows. I also looked at the widely used, industry-standard NaCl library which does provide a serialization (wire) format for cryptographic keys which are encrypted.

Lastly, I recognize a need for providing private key material for usage in test vectors. This is a fairly common practice to ensure that cryptographic operations (sign/verification, encrypt/decrypt, etc.) are consistent across implementations given static input values. I believe we should provide private key JWK representations using the secretKeyJwk property defined by VC-JOSE-COSE for usage in test vectors only.

To summarize:

1. We should follow the best practice of not moving private keys as much as possible.
2. Portable DID should exist and contain all but private key material, which should be swapped with an interface to KeyAccess.
3. We should implement common secure key access and storage patterns (e.g. AWS KMS, AWS Secret Manager, secure memory) and provide a path for sourcing requirements for other key access mechanisms should they arise for us or our customers.
4. We should provide secretKeyJwk properties in test vectors where it is important to verify and maintain consistency between our SDK implementations.

[KendallWeihe]

What's being proposed is a bearer DID.

The value of a portable DID in relation to a bearer DID is not the document and associated metadata (because bearer DIDs have the document and could include metadata too), the value is in the private keys. If we remove private keys from the portable DID then we should remove the concept of a portable DID altogether and instead define an implementation standard of serializing/deserializing bearer DIDs.

I agree with the security concerns, so I'm not disagreeing with your comment @decentralgabe but trying to elucidate how that maps onto implementation. Basically if we keep portable DIDs but remove the private keys then we'll have to carry the baggage of portable DIDs which is an unnecessary abstraction into/out-of bearer DIDs.

A lot of this boils down to the lack of standardization across key management systems. I feel like we resulted to portable DIDs because we needed an easy way to move keys around across all of our implementations. @decentralgabe I know the answer to this (probably "no") but I have to ask, should we consider adding some sort of KeyAccess interface (as you put it) to the web5-spec itself?

[decentralgabe]

we should remove the concept of a portable DID altogether and instead define an implementation standard of serializing/deserializing bearer DIDs

Do we have a use case for serializing/deserializing bearer DIDs? If we do, then I agree with defining an encrypted format similar to how ETH keyfiles work.

should we consider adding some sort of KeyAccess interface (as you put it) to the web5-spec itself?

This seems like useful guidance worth specifying.

[KendallWeihe]

I'm now seeing how this is all downstream of the surface area of key management across the N-number environments we have to account for

@decentralgabe do you agree? If so, perhaps step one is to audit all the various environments we support and to understand the options & constraints in relation to key management. Perhaps we should close this ticket as won't-do and open a new ticket for aggregating analysis around key management? and we should also be more concrete as to what we think is relevant to specification and/or docs/guides.

I think this may be an excellent use case for a core rust library

[decentralgabe]

I'm now seeing how this is all downstream of the surface area of key management across the N-number environments we have to account for

Yes I agree.

If so, perhaps step one is to audit all the various environments we support and to understand the options & constraints in relation to key management. Perhaps we should close this ticket as won't-do and open a new ticket for aggregating analysis around key management?

I like this suggestion and believe we should follow it.

For test vectors I prefer we make available a few common JWKs that we can use across multiple tests.