feat: ipld schema for representing UCANs in CBOR

[Gozala]

Following up on our UCAN IPLD representation call and submitting IPLD schema currently used by https://github.com/ipld/js-dag-ucan

[Gozala]

@oed, @ukstv I propose raising issues inline here, specifically about missing fields etc..

[oed]

Thanks @Gozala, Will let @ukstv weigh in with more specific comments.

[ukstv]

@Gozala So, I made a comparison at https://github.com/ukstv/ucan-codec-comparison That includes vanilla dag-ucan encoding, one with did:key as strings, and one with shortened field names (they all contribute to block size).

Note: The text is also available as https://hackmd.io/@ukstv/H1SxICj8q/edit

Conclusion: Semantically dag-ucan and UCAN-CACAO encoding are the same. There is a small constant difference in size. Using the present code and specifications, for the same UCAN, we have 325 bytes for CACAO and 295 bytes for dag-ucan. If we modify dag-ucan to use did:key strings, we get 339 bytes. After modification of the format to further reduce number of bytes, we get 317 bytes for CACAO and 298 bytes for dag-ucan, with the constant difference between the formats.

Overall, CACAO and dag-ucan are comparable in size, with constant difference due to richer format of CACAO metadata. In the world where JWT is the only option, it makes sense to go with dag-ucan. If we have interoperability in mind though, it makes sense to stick with CACAO.

[expede]

👀

[Gozala]

Thanks @ukstv for doing this comparison, it is helpful to have this data. I think I may have not been very clear in the call though when I questioned reasoning for having those static fields, it's adds (although as your analysis show insignificant) overhead to the block size, but that is not the only overhead & I'm personally more worried about complexity overhead. It all may be well justified, yet I would really like to:

1. Understand that justification
2. Asses if there are other, better ways to meet desired goals.

My current understanding of the issue is as follows, please correct me where I'm wrong:

1. CACAO uses specific format that is different from dag-ucan.
2. UCAN-CACAO has additional header / signature metadata because CACAO supports non-ucan payloads.
3. If we want to allow interop with CACAO that extra bit of metadata is needed to signal CACAO implementation what to do.

Assuming my statements are accurate (to a reasonable degree), I see several other ways we could address the issue without introducing following overhead:

1. Few extra bytes + unreadable names.
2. Greater possibility to represent invalid UCANs ( When this things are derived simply from issuer no such errors are possible and that is why dag-ucan intentionally leaves those out)
   • What should implementation do if header is missing ?
   • What should implementation do if s/t isn't JWT ?
   • What should implementation do if issuer did does not match s/m/alg ?
   • What should implementation do if issuer isn't did:key: ?
3. What to do with proofs that are valid CACAO but invalid UCAN ? If answer is just fail then
   • I see benefits on CACAO side of things but fail to recognize them on UCAN side. I would much rather seek ways to achieve true interop instead.
   • I also see different ways CACAO can achieve the same effect (more on that later) without introducing this overhead into UCANs.

Here are few alternative options by which CACAO could achieve compatibility without introducing outlined tradeoffs (although non are tradeoff free):

1. Use alternative signaling mechanism. E.g. dag-ucan uses CID codec code to signal whether UCAN is in JWT or CBOR format.
   • @oed correctly called out that deploying different codecs across network can be challenging.
2. Signal with mutiformat code inside the block (E.g add code field into data model as opposed to CID)
3. Signalling through envelope structures (IPLD schema even has first class support for this)
   • Instead of adding additional metadata into UCANs, CACAO can simply wrap UCANs into structure with additional metadata (in fact dag-ucans schema and UCAN-CACAO payload schema are pretty similar so this seems like a more natural path to explore)
4. I assume that CACAO implementation has a lot more going on beyond CBOR. That suggests it should not be too difficult to support multiple schemas at that layer (as long as it can differentiate between them)
   • That is something js-dag-ucan does as well. It has two View classes implementing same interface and abstracting between alternative data model schemas (JWT vs CBOR). Could CACAO do the same and have the header and other metadata be derived by the view ?
   • I would also be in favor of replacing version field with schema field that links to IPLD schema, which can provide means for signalling.

I am sure I still lack a lot of context, but that is also precisely why I think it would make sense to capture desired goals and limitations of current UCAN spec. With that we could collaboratively explore best ways to address them and potentially find better outcomes, if nothing else arrive to shared understanding. Maybe we could even address limitations in UCAN spec that motivated CACAO in first place (or at least partially so) ?

I want to stress that I'm not against adding additional fields and outlined overhead (in size & complexity), however I would like us to weight other options and have clear understanding what the benefits are.

[ukstv]

Let's start from the very beginning: why CACAO in the first place.

We are working on a capability-based system.

Desideratae:

1. should work with existing blockchain wallets:
   • should support wildly used blockchain signature schemes,
2. should be serialized to IPLD record:
   • small size,
   • deterministic encoding.
3. should be extendable:
   • signal payload semantics.

The main question is how to represent a capability. It informs how we design API around it and transmit the capabilities around. There are two cool formats to represent capabilities - zCAP-LD and UCAN. zCAP-LD is based on JSON-LD, with drawbacks like LD-Proof algorithms (problematic!), JSON-LD overhead for Linked Data fields, and weird semantics of a capability lifecycle. UCAN has proper semantics (great work indeed), and builds on JWT (everybody loves it). Let's focus on that.

JWT specification tells us how to prepare a signing input (base64url of header and payload), and it provides a set of available algorithms (ES256K, ES256, EdDSA,...) for signing. The first goal for us is to turn "Sign-in With Ethereum" (SIWE) into a capability, the next goalpost is to support other blockchains (SIWx). SIWE/SIWx uses certain format for the payload, and a peculiar signing scheme. A user signs a string payload (instead of a carefully crafted signing input) with blockchain wallet. It implies using algorithms not present in JOSE Algorithms registry; corollary: it is incompatible with JWT [there is an option to push another algorithm to IANA, but it would not really help]. Using string payload for signing prevents us from using JWT-specified procedure to prepare a signing input (=base64url of header and payload), which is again incompatible with JWT.

All right, one could argue, let's agree to use JWT with custom non-IANA alg to specify custom signature algorithm [serves desideratum 1], and maybe additional header field to signal payload semantics [serves desideratum 3], like UCAN does. Also, we mandate canonicalization. At this point we have familiarity of JWT, but no compatibility with existing tooling.

One of the benefits of JWT is serialisability. We have simple way to pass a JWT in compact form (=dot-delimited base64url-encoded components) through a string-based transport like HTTP. Yet, it is a single JWT that could be easily passed. When multiple JWTs have to be transmitted, we have to create some additional mechanism. With regards to IPLD representation [desideratum 2], familiar JWT serialization form brings zero benefits, as witnessed by ADL in dag-ucan.

By the end of the day, we have no compatibility with existing tooling [even if necessary algorithms are passed through IANA, JWS mandates signing over signing input, see RFC7515 section 5.1], useless serialization format, and only confusing resemblance to JWT. So, why even drag this container when it hurts more than benefits? At this point, I hope, the rationale behind CACAO is clear.

Now, let's zoom out a bit. Here we all are working towards a goal of having a harmonious capabilities system that works in web2 (UCAN realm) and web3 (CACAO realm).

For this we better have a single IPLD container. We are early here.

As per your suggestion, it makes sense to do a union IPLD schema. I would suggest a union schema to save precious bytes. Rough scetch below to see if you are all right with exploring it further:

type Capability union {
    | UCAN_0_9_0 "ucan@0.9.0"
    | EIP_4361 "eip4361"
} representation inline {
    discriminantKey "t"
}

type UCAN_0_9_0 struct {
    issuer SigningKey
    audience SigningKey
    signature Signature
    capabilities [Capability]
    expiration Int
    -- ...
}

type EIP_4361 {
    p Payload
    s Signature
    -- ...
}

This way we can represent both existing formats in a future proof fashion. We can add more types via free-to-specify t discriminant. Of course, the schema, if wrapped in a proper library, would have to export these as a separate views.

Three things I would like to mention here.

Field Names While readability is important, I would suggest to use shorter names. What we design here is not indended for direct human consumption. Even the glorious JWT as per RFC7519 is compact, URL-safe format, which is far from human-readable. We do optimise for size and ease of programmatic access, as long as it does not interfere with the size goal. I talked to a friend of mine recently, who is in web3 now and have solid background in web2. He was surprised dag-ucan IPLD representation uses longer fields which contribute to size increase, and also require mental gymnastics to translate between longer names and an accepted, standardized, familiar three-letter JWT claims.

Signalling While having a dedicated codec might be a good idea, let's not forget that it might take a while. A capability is not something used standalone. Usually context dictates that a CID references a capability. In Ceramic case, we require that cap field in a dag-JWS header references a capability. In some other context, people use a special Invocation concept, which is based on JWS. It too references a capability via cap header. So, signalling "it is a capability!" is not an immediate problem we need to solve, I believe.

Correctness A library on developer's side might be helpful in avoiding silly mistakes by providing exquisite developer experience. We should be prepared to a fact though, that a message coming from an external source is not always valid. Nothing prevents a malicious or just buggy code to post an invalid payload, be it vanilla JWT or UCAN or CACAO. What we are trying to achieve here is to specify how a valid capability should look like, leaving questions of dummy-proof APIs and enforcement of constraints to some other venue.

[Gozala]

Field Names While readability is important, I would suggest to use shorter names. What we design here is not indended for direct human consumption. Even the glorious JWT as per RFC7519 is compact, URL-safe format, which is far from human-readable. We do optimise for size and ease of programmatic access, as long as it does not interfere with the size goal. I talked to a friend of mine recently, who is in web3 now and have solid background in web2. He was surprised dag-ucan IPLD representation uses longer fields which contribute to size increase, and also require mental gymnastics to translate between longer names and an accepted, standardized, familiar three-letter JWT claims.

I'm not opposed to shorter names, they were originally based on names used in ts-ucan library, and I was hoping that descriptive names with tools like https://explore.ipld.io/ would be of benefit. That said I'm open to short names. That said I'm personally not into single character names, yet I can buy into this if everyone feels extra bytes are worth it.

Signalling While having a dedicated codec might be a good idea, let's not forget that it might take a while. A capability is not something used standalone. Usually context dictates that a CID references a capability. In Ceramic case, we require that cap field in a dag-JWS header references a capability. In some other context, people use a special Invocation concept, which is based on JWS. It too references a capability via cap header. So, signalling "it is a capability!" is not an immediate problem we need to solve, I believe.

I'm not sure I understand the point you're trying to make here.

Correctness A library on developer's side might be helpful in avoiding silly mistakes by providing exquisite developer experience. We should be prepared to a fact though, that a message coming from an external source is not always valid. Nothing prevents a malicious or just buggy code to post an invalid payload, be it vanilla JWT or UCAN or CACAO. What we are trying to achieve here is to specify how a valid capability should look like, leaving questions of dummy-proof APIs and enforcement of constraints to some other venue.

I'm not sure I fully agree on this point. I personally find that making impossible states impossible is just a good design. Fact that you may encounter malicious or malformed data is not a good reason to create more room for logically incorrect data, if nothing else it creates more room for attacker to find exploit.

[Gozala]

@ukstv we talked a bit about this in our last call, but I just like to capture some of it here for further discussion. We have plans to support "sign in with wallet" use case, however instead of extending UCANs with other DIDs and signing mechanism we plan on doing it within the current spec limits. Specifically "capabilities" in UCANs are pretty open ended:

1. You can stick arbitrary data along along the side of can and with fields.
2. Capability validator / executor is responsible for interpreting capabilities themself and can bring any code to do so.

With that perspective instead of using blockchain account as token issuer, we plan on delegating capabilities from blockchain account to a specific did:key. To do so you we can encode arbitrary capability e.g. as follows:

{
   can: "siwe/delegate"
   with: "did:ethr:0xf3beac30c498d9e26865f34fcaa57dbb935b0d74"
   capability: { ... }
   signature: "..."
}

Idea here is that account holder "did:ethr:0xf3beac30c498d9e26865f34fcaa57dbb935b0d74" delegated specific capability to e.g. did:key:zAlice and provided signature from account holder to prove it.

Please note that going about this way has following properties:

1. Requires no changes to UCAN spec.
2. No need to support any DIDs other then did:key.
3. No need to support new signing algorithms when verifying UCANs in middleware.
   • Executor still needs additional code to validate provided proof but that seems unavoidable, at least we reduce places where that needs to happen.
4. Validators / executors not concerned with siwe/delegate capability will not encounter errors when trying to fetch / decode proofs that use different DID method or signing algorithm.

Overall it seems like a better way to bring interop between systems, that does not require bringing various fields from each other. What do you think ?

[oed]

@Gozala I think the approach of putting a SIWE message in an attenuation as you suggest makes sense. This was the approach we discussed in Amsterdam. The main modification I would suggest is to actually just put the CID of a CACAO object in the attenuation. That way you can leverage the growing set of CACAO tooling for SIW(X) when verifying siwe/delegate.

[Gozala]

The main modification I would suggest is to actually just put the CID of a CACAO object in the attenuation. That way you can leverage the growing set of CACAO tooling for SIW(X) when verifying siwe/delegate.

That sounds better indeed (my example was just a sketch)! It would be cool to see a more concrete example of what that might look like along with what tooling could be leveraged 🤩

[expede]

I expanded the text, made a couple of tweaks, and moved the text to its own spec: https://github.com/ucan-wg/ucan-ipld/pull/1

Why it's own spec? It's a best practice recommended by the IPR template that the UCAN WG uses.

Let me know what you think over there!

[oed]

@Gozala We have an implementation of SIWE (and experimental SIWSolana) for CACAO over here: https://github.com/ceramicnetwork/cacao/

[expede]

Closing in favour of https://github.com/ucan-wg/ucan-ipld/pull/1