Open tplooker opened 2 months ago
+1 for render methods as a means to brand a credential. BC Gov has successfully demonstrated this by applying OCA bundles to anoncreds VC's as a branding overlay. We are now working on a OverlayCaptureBundle
renderMethod
in partnership with the Swiss government for json-ld VC's.
Nice! Agree on the use of renderMethod
as a credential-focused, protocol-independent mechanism for this purpose. Hopefully your work will show-up as part of https://w3c-ccg.github.io/vc-render-method/
I question why we need to map from the property's URI to a compact string to a render string. Why do we need two mappings, especially when JSON-LD caters for languages?
I question why we need to map from the property's URI to a compact string to a render string. Why do we need two mappings, especially when JSON-LD caters for languages?
There continues to be confusion in this thread on the layers and what each layer is for. Let's take an example:
"title": "Manager"
If you don't know the context of the message, the expression above, on it's own, might as well be:
"p28dv4ja91": "Manager"
The mistake that keeps being made in some interpretations in this thread is that you can guess as to what the property means/is just by reading it as an English word (the security disclosure makes this same mistake). IOW, people are presuming the context just because it looks like an English word, but that's an insecure way to do processing in any information system. Two party systems establish context implicitly (depending on the two parties communicating). Three party systems establish context explicitly (because you can't scale past point-to-point, two-party integrations, without that happening).
Now, you can add a context like this:
"https://vocab.example/title": "Manager"
and you start getting to something that's less ambiguous, there's a global identifier that identifies the term, and if you use JSON-LD, you can get it into a developer-friendly form:
"@context": "https://vocab.example/v1",
"title": "Manager"
So, the developer can start plugging title
into their code to retrieve the value, but that DOES NOT mean they can directly expose that text to a UX (without some extensive vetting of the context). Again, this is the mistake made throughout a number of comments in this thread... presuming that just because you know what the English word "title" means, that you know what the value expresses. Is it a Book Title? Or a Job Title? Or maybe it's a Website Title?
So, you have to understand the context in order to process the message. You cannot just expose "title" to a UI and presume that the person looking at the screen is going to come away with the proper understanding. The English word and the semantics probably align 95% of the time, but there will be cases where you'll be wrong and those cases might matter because they go into making a critical decision.
IOW, the mapping from a property URI (https://vocab.example/title
) to a compact string (title
) is a tool that is useful for developers when writing code, as long as they understand the context.
How that property URI is rendered to a non-developer in a UI is an entirely different thing. It could leverage the vocabulary, that might express one or more human-readable strings in different languages, to render the property name to the screen. Or it could leverage a render method that does the same sort of thing, but in a more textually, aurally, or visually compelling way.
Why do we need two mappings, especially when JSON-LD caters for languages?
As highlighted above, we can leverage JSON-LD's language features for classes, properties, and values, but we need to make sure everyone understands the distinction between URL->term compaction, and who that benefits (the developer), vs. URL->display value translation and who that benefits (non-developers).
@aniltj we are planning for it!
the point I am trying to make is that the property is "https://vocab.example/title" (but it could be" https://a.c.c.com/xyz", it does not really matter.) Computers can process this URL as easy as ABC or XYZ, they don't need a compact form. So why not render directly from "https://vocab.example/title" to whatever you want to display to the user. There is no point in having the compact encoding in between. Or to put it another way, why not have a context that maps directly to the rendering so that the VC becomes whatever you want to display to user, such as "ABC Company Title": "manager"
TL;DR: The ideal and most scalable solution to displaying VCs is "render methods". I recommend working with the community to help develop these. Anything else is a stop gap measure and should not drive the core design of VC primitives. A variety of these stop gap measures are available today, some being better than others depending on the situation.
It appears my specific use case has become a bit of a distraction from the point I was trying to make, which wasn't around how one could solve credential branding in different ways and to be clear I don't believe any of the options presented in response would have actually solve our issue in this instance. The simple reality is, there was no standard at the time and all we wanted was to be able to extend the VC data model in a way that didn't cause the pain it caused to downstream applications that simply wanted to ignore the branding information.
What I was trying to illustrate is DI based VC's make all @context
entries critical for verifier applications to know and trust, even when some @context
entries are not critical for all verifiers to understand. This is what creates the scaling issue, forcing systems to load and manage redundant information creates friction for no value and only acts as an impediment.
I do have many other usecases that demonstrate this problem, however I am reluctant to share in this thread as it will just get buried in what is already a difficult conversation to follow. Suffice to say the problem is not about credential branding it is a generic problem that affects many use cases.
I'll try and respond below to some of the suggested solutions to this problem
@dlongley wrote
This only leaves verifiers that are not willing to load contexts from the Web, an intentionally more inflexible subset of verifiers.
This appears as a direct contradiction with the guidance in the data integrity specification, which is as follows:
Implementations MAY load application-specific JSON-LD context files from the network during development, but SHOULD permanently cache JSON-LD context files used in conforming documents in production settings to increase their security and privacy characteristics
Either production systems SHOULD or SHOULD NOT resolve contexts files from the network, you appear to be calling systems that fall into the latter "inflexible", rather then just following the specification guidance.
@dlongley wrote
But don't use a catch-all "issuer-dependent"
@vocab
, because it doesn't play nicely with others, creates confusion, and incentivizes behavior that leads to more trouble in the ecosystem.
This is at odds with permission-less innovation, in my opinion its a round about way of saying "all extensions to the VC data model must be blessed by this standards community in order to work well".
@dlongley wrote
Verifiers such at this can announce the contexts that they are willing to accept, when requesting VCs. Providers can run the JSON-LD compaction API to recompact a VC to their accepted context. You might even be able to use selective disclosure to remove some terms, if the verifier allows it. This can even help get interoperability with verifiers that simply won't accept any extra terms at all (e.g., JSON schema "additionalProperties": false).
Im surprised to hear that a verifier sending a list of all the contexts it supports to a wallet is considered a solution to this problem. Aside from the fact that this would significantly bloat the size of the average credential request, the wallet would then need to interpret this list and performing checks like
@dlongley wrote
Express the VC using a context you believe they will accept before sending it to them. Again, you might even be able to use selective disclosure to remove some terms, if the verifier allows it.
Again my use case was specifically for when this isn't the case, the context isn't known or trusted to the relying party nor is it required for them to trust it. It appears your solution here in the second sentence would require some advanced tree shaking capability to remove @context
entries that aren't used in the selective disclosure case, for this to work and how one achieves this with JSON-LD architecturally seems quite far fetched.
@dlongley wrote
Compact to your special context when using your special terms (that are for your software only anyway), keeping your vendor-specific terms expressed as full URLs otherwise.
So not actually using the compact JSON-LD form in this case (at least for these terms)? Again this assumes the wallet knows which contexts the verifier knows about ahead of time which seems very difficult to establish.
(edited to put backticks around @words
)
@tplooker,
@dlongley wrote
This only leaves verifiers that are not willing to load contexts from the Web, an intentionally more inflexible subset of verifiers.
This appears as a direct contradiction with the guidance in the data integrity specification, which is as follows:
Implementations MAY load application-specific JSON-LD context files from the network during development, but SHOULD permanently cache JSON-LD context files used in conforming documents in production settings to increase their security and privacy characteristics
Either production systems SHOULD or SHOULD NOT resolve contexts files from the network, you appear to be calling systems that fall into the latter "inflexible", rather then just following the specification guidance.
Ah, I can see where some confusion is coming from. Your application must only consume terms from contexts it understands and those are the contexts that should not, in general, be loaded from the Web. If a document arrives with some other context on it, then you need to recompact it to one you do understand (or reject it). The former can be loaded from the Web, the latter should not be.
So, the advice here is for contexts from which terms are actually being consumed by the application. This does not apply to contexts that will be replaced during transformation via the JSON-LD compaction API. We could probably tweak this text a little bit to help implementers understand the difference.
@dlongley wrote
But don't use a catch-all "issuer-dependent"
@vocab
, because it doesn't play nicely with others, creates confusion, and incentivizes behavior that leads to more trouble in the ecosystem.This is at odds with permission-less innovation, in my opinion its a round about way of saying "all extensions to the VC data model must be blessed by this standards community in order to work well".
No, this is just saying that if you want to interoperate with others, you should use globally unambiguous terms, do not rely on @vocab
in the above way. You do not need permission to use globally unambiguous terms. I'll note that this is similar to advice given to users of JWTs via RFC 7519: "private claims" should be used with caution as they are subject to collision. I also mentioned a number of other problems with using them in my comments.
@dlongley wrote
Verifiers such at this can announce the contexts that they are willing to accept, when requesting VCs. Providers can run the JSON-LD compaction API to recompact a VC to their accepted context. You might even be able to use selective disclosure to remove some terms, if the verifier allows it. This can even help get interoperability with verifiers that simply won't accept any extra terms at all (e.g., JSON schema "additionalProperties": false).
Im surprised to hear that a verifier sending a list of all the contexts it supports to a wallet is considered a solution to this problem. Aside from the fact that this would significantly bloat the size of the average credential request, the wallet would then need to interpret this list and performing checks like
Sorry for any confusion here, I did not mean to suggest that every context a verifier could possibly support would be sent in every request. Instead, only the contexts that are specific to the query being asked would be included. This is already expected anyway -- and how some software works today. I've included a short example below in response to your other comments that I hope helps.
@dlongley wrote
Express the VC using a context you believe they will accept before sending it to them. Again, you might even be able to use selective disclosure to remove some terms, if the verifier allows it.
Again my use case was specifically for when this isn't the case, the context isn't known or trusted to the relying party nor is it required for them to trust it. It appears your solution here in the second sentence would require some advanced tree shaking capability to remove
@context
entries that aren't used in the selective disclosure case, for this to work and how one achieves this with JSON-LD architecturally seems quite far fetched.
No, advanced tree shaking capability should not be required, rather, just call the JSON-LD compaction API.
As a quick example, suppose there is a common context for international driver's licenses and an uncommon context for localized terms, like for US-only driver's licenses. As far as I can tell, I'm constructing this according to your use case -- as the verifier (aka relying party) does not "know or trust" the US-only context.
So, in this case, the user has a US driver's license with the core VC v2 context core
, the international context A
and the US context, B
. Let's say it expresses that the user has a driver's license with an internationally recognized term like "documentnumber" and some US-only "aamva*" terms, like so:
{
"@context": [core, A, B],
...,
"credentialSubject": {
"driversLicense": {
"document_number": "123456789",
"aamva_foo": "some-US-only-field-value",
...
}
},
...
}
The verifier only trusts the contexts core
and A
and it wants the user's "document_number" for their driver's license. So they ask for it like this (notice they include the contexts they understand):
{
"query": {
"type": "QueryByExample",
"credentialQuery": {
"example": {
"@context": [core, A],
"credentialSubject": {
"driversLicense": {
"document_number": ""
}
}
}
}
},
...
}
Now the wallet performs matching (implementation specific) to get a matching VC, perform selective disclosure to produce a VC with just the field desired -- and then pass that to the JSON-LD compaction API with {"@context": [core, A]"}
(from the verifier's query) as the context to recompact to. The output is now acceptable to the verifier (it will not contain context B
because the wallet called standard W3C APIs).
@dlongley wrote
Compact to your special context when using your special terms (that are for your software only anyway), keeping your vendor-specific terms expressed as full URLs otherwise.
So not actually using the compact JSON-LD form in this case (at least for these terms)? Again this assumes the wallet knows which contexts the verifier knows about ahead of time which seems very difficult to establish.
JSON-LD compacted form is still being used here. It's just that the only context is the VC v2 core context; it just doesn't have your extra context when you're sharing it with others. There might have been more confusion on this point, hopefully this explanation helps. The wallet can always assume the verifier knows at least the VC v2 core context, knowing this context is a hard requirement in the spec. Other contexts could be communicated in a number of ways, including like in the example above.
(edited to put backticks around @words
)
@dlongley wrote
Verifiers such at this can announce the contexts that they are willing to accept, when requesting VCs. Providers can run the JSON-LD compaction API to recompact a VC to their accepted context. You might even be able to use selective disclosure to remove some terms, if the verifier allows it. This can even help get interoperability with verifiers that simply won't accept any extra terms at all (e.g., JSON schema "additionalProperties": false).
IMO, not "might", but I rather think you must, since without some sort of SD, this approach does not work for verifiers that don't want to fail if the secured document has terms from additional context they don't know and care about.
It means, you would need to use either BBS, or ECDSA-SD. Otherwise, the verifier would not be able to verify the signature.
@dlongley wrote
Compact to your special context when using your special terms (that are for your software only anyway), keeping your vendor-specific terms expressed as full URLs otherwise.
What JSON-LD has effectively done is to replace an IANA central registration authority for compact terms with thousands of local registration authorities, which no implementation can feasibly trust. A few will be trusted, like the one published in the VC DM, or in another ISO or IETF standard or at national or international level. The rest will only be trusted by a small subset of entities. Which means that all implementations in general will need to discard all the non-trusted contexts and display the full URIs to the user (because they wont trust any render method for a context they don't trust). This solution will remove the bugs identified by @tplooker and should close this issue. I suggest that we add text to the DI spec to describe this, and move on.
@dlongley wrote
Compact to your special context when using your special terms (that are for your software only anyway), keeping your vendor-specific terms expressed as full URLs otherwise.
What JSON-LD has effectively done is to replace an IANA central registration authority for compact terms with thousands of local registration authorities, which no implementation can feasibly trust. A few will be trusted, like the one published in the VC DM, or in another ISO or IETF standard or at national or international level. The rest will only be trusted by a small subset of entities.
This is essentially the same situation with regular JSON. IANA is one option but nobody prevents anybody from using distributed registries that define JSON property names that are collision-resistant such as using reverse domain names for instance or full URIs. JSON-LD in VCDM uses a similar approach with the difference that term names are compacted to context, so they look nicer which can cause some trouble and EDIT: has implementation considerations/limitations and operational assumptions as discussed in this thread. Especially requiring all context to be known, understood and trusted EDIT: even for those terms a verifier does not care about (as mentioned above), to verify DI, is problematic for the reasons @David-Chadwick just pointed out.
Not sure if helpful, but I tried to go through the VCDM/DI specs to identify problematic or not helpful language.
Ensure that all values associated with a
@context
property are in the expected order, the contents of the context files match known good cryptographic hashes for each file, and domain experts have deemed that the contents are appropriate for the intended use case.
This section is non-normative which might be problematic.
Securing mechanism specifications MUST provide a verification mechanism that returns only the information in the conforming document that has been secured, without any securing mechanism information included, such as proof or JOSE/COSE header parameters and signatures.
For any any LD-DI crypto suite, the verifiedDocument
is the unsecureDocument
(including @context
) if the signature was verified. It seems odd to me that LD-DI still returns @context
in case the attack described in this issue was performed. Wouldn't it be more appropriate to return no @context
?
Securing mechanism specifications MUST provide [...] A document that only contains information that was successfully secured.
Since the @context
is not secured, shouldn't a DI proof not include that information (see above)?
Also to @David-Chadwick's point , perhaps it would be more appropriate to return the expanded document in that case?
Securing mechanism specifications SHOULD provide integrity protection for any information referenced by a URL that is critical to validation.
I know it is a SHOULD and not a MUST, but this means Data Integrity Proofs don't adhere to this requirement, since DI proofs don't protect the integrity of @context
entries.
I'm also generally missing language in DI proofs that says anything about validation of SRI-protected URLs.
Why is this requirement even included in the spec if none of the LD-DI proofs make use of that?
Another thing I noticed is that people look at VCDM 2.0 + DI mostly as one unit, and disregard that DIs can be used with any other document as well. It seems to me that a lot of language that makes DI securely usable is in VCDM 2.0, not in DI (e.g., SRI, @context
considerations). Just from a security and layering point of view I don't like this approach, since DI should be a standalone spec that works with any JSON-LD doc securely.
Another thing I noticed is that people look at VCDM 2.0 + DI mostly as one unit, and disregard that DIs can be used with any other document as well. It seems to me that a lot of language that makes DI securely usable is in VCDM 2.0, not in DI (e.g., SRI,
@context
considerations). Just from a security and layering point of view I don't like this approach, since DI should be a standalone spec that works with any JSON-LD doc securely.
That might become a little more complicated insofar as the DI spec should be usable, in theory, for RDF datasets at large, not only when they are serialized in JSON-LD... But I take your point.
The sketched solution here https://github.com/w3c/vc-data-integrity/issues/272#issuecomment-2184187798 is only one of the many, the simplest one - to prevent consuming data you don't trust by simply rejecting an unknown context set. There are other solutions, even on RDF level, that could be applied to prevent this issue from happening - with pros/cons (robustness, complexity, etc.)
The outcome of this discussion should be something like: Do not ever consume data you don't trust.
@awoie,
IMO, not "might", but I rather think you must, since without some sort of SD, this approach does not work for verifiers that don't want to fail if the secured document has terms from additional context they don't know and care about.
It means, you would need to use either BBS, or ECDSA-SD. Otherwise, the verifier would not be able to verify the signature.
While it is generally better to use SD for data minimization, it isn't true that you have to use it here. It would work just as well to compact a non-SD VC to the context the verifier supplies. The additional terms would just appear as full URLs (and be ignored) when recompacting to their context, as expected.
@David-Chadwick wrote:
There is no point in having the compact encoding in between.
Which completely ignores the point about the compact encoding being useful for developers. :)
This solution will remove the bugs identified ... I suggest that we add text to the DI spec to describe this, and move on.
There are 11 solutions under consideration at the moment, which one of those do you mean by "This solution"... and what text do you propose to "describe this" given where the discussion is, currently?
@awoie wrote:
since without some sort of SD, this approach does not work for verifiers
There were multiple solutions proposed that don't depend on some sort of selective disclosure and shows how there are multiple ways that the current approach scales.
@David-Chadwick wrote:
There is no point in having the compact encoding in between.
Which completely ignores the point about the compact encoding being useful for developers. :)
I'm not sure it's actually clear that this is the case, at least not from what's written there.
If we compare the form that I believe is being asserted to be developer friendly:
"@context": "https://vocab.example/v1",
"title": "Manager"
and another form mentioned:
"https://vocab.example/title": "Manager"
As a developer, to me, the latter looks simpler to parse and infer meaning from. Did I misunderstand something?
@jogu
There is no point in having the compact encoding in between.
Compacted version could be useful to some, e.g. to help with adoption/integration.
Iron VC (an open-source library I work on) uses fully expanded IRIs when parsing a VC internally, but gives an option to get a compacted version of the same VC and it's left up to a developer to choose what fits the best.
Actually, mapping an expanded VC to a class instance can be seen as a compaction where a context is defined by class name.
@msporny Here is some straw man text for consideration in the DI text
DI signature verifiers MUST trust all the @context
definitions in the verifiable credential. Globally there are expected to be thousands of @context
definitions, and no implementation can feasibly trust all of these definitions. A few definitions will be widely trusted, such as the one published in the VC DM, or in another ISO or IETF standard or by a national or international authority. The remainder are likely to be only be trusted by a small subset of implementations. Consequently all implementations MUST discard all the non-trusted @context
definitions and display the full URIs to the human (developer, implementor or end user).
@jogu wrote:
As a developer, to me, the latter looks simpler to parse and infer meaning from. Did I misunderstand something?
Perhaps? You seem to be arguing against the way that the vast majority of most HTTP APIs that return JSON objects work today. I'm struggling to think of a popular HTTP API, or developer API for that matter, that specifies all of its objects using URLs for all properties and type names.
To put a finer point on it, you seem to be arguing that VCs use these sorts of objects:
{ "@type": [ "https://www.w3.org/2018/credentials#VerifiableCredential", "https://w3id.org/vdl#Iso18013DriversLicenseCredential" ], "https://schema.org/description": "A license granting driving privileges in Utopia.", "https://schema.org/image": { "@id": "data:image/jpeg;base64,iVBORw0KGgoAAAANSUhEUg...kSuQmCC" }, "https://schema.org/name": "Utopia Driver's License", "https://www.w3.org/2018/credentials#credentialSubject": { "@id": "did:example:12347abcd", "@type": "https://w3id.org/vdl#LicensedDriver", "https://w3id.org/vdl#license": { "@type": "https://w3id.org/vdl#Iso18013DriversLicense", "https://w3id.org/vdl#birthDate": { "@type": "http://www.w3.org/2001/XMLSchema#dateTime", "@value": "1998-08-28" }, "https://w3id.org/vdl#documentNumber": "542426814", "https://w3id.org/vdl#drivingPrivileges": { "@type": "@json", "@value": [ { "codes": [ { "code": "D" } ], "vehicle_category_code": "D", "issue_date": "2019-01-01", "expiry_date": "2027-01-01" }, { "codes": [ { "code": "C" } ], "vehicle_category_code": "C", "issue_date": "2019-01-01", "expiry_date": "2017-01-01" } ] }, "https://w3id.org/vdl#expiryDate": { "@type": "http://www.w3.org/2001/XMLSchema#dateTime", "@value": "2028-08-27T12:00:00-06:00" }, "https://w3id.org/vdl#familyName": "TURNER", "https://w3id.org/vdl#givenName": "SUSAN", "https://w3id.org/vdl#issueDate": { "@type": "http://www.w3.org/2001/XMLSchema#dateTime", "@value": "2023-01-15T10:00:00-07:00" }, "https://w3id.org/vdl#issuingAuthority": "UADMV", "https://w3id.org/vdl#issuingCountry": "UA", "https://w3id.org/vdl#portrait": { "@id": "data:image/jpeg;base64,/9j/4AAQSkZJR...RSClooooP/2Q==" }, "https://w3id.org/vdl#sex": { "@type": "http://www.w3.org/2001/XMLSchema#unsignedInt", "@value": 2 }, "https://w3id.org/vdl#unDistinguishingSign": "UTA", "https://w3id.org/vdl/aamva#akaSuffix": "1ST", "https://w3id.org/vdl/aamva#familyNameTruncation": "N", "https://w3id.org/vdl/aamva#givenNameTruncation": "N" } }, "https://www.w3.org/2018/credentials#expirationDate": { "@type": "http://www.w3.org/2001/XMLSchema#dateTime", "@value": "2028-11-15T12:00:00-06:00" }, "https://www.w3.org/2018/credentials#issuanceDate": { "@type": "http://www.w3.org/2001/XMLSchema#dateTime", "@value": "2023-11-15T10:00:00-07:00" }, "https://www.w3.org/2018/credentials#issuer": { "@id": "did:key:z6MkjxvA4FNrQUhr8f7xhdQuP1VPzErkcnfxsRaU5oFgy2E5", "https://schema.org/image": { "@id": "https://dmv.utopia.example/logo.png" }, "https://schema.org/name": "Utopia Department of Motor Vehicles", "https://schema.org/url": { "@id": "https://dmv.utopia.example/" } } }
Instead of these sorts of objects:
{ "@context": [ "https://www.w3.org/2018/credentials/v1", "https://w3id.org/vdl/v2", "https://w3id.org/vdl/aamva/v1" ], "type": [ "VerifiableCredential", "Iso18013DriversLicenseCredential" ], "issuer": { "id": "did:key:z6MkjxvA4FNrQUhr8f7xhdQuP1VPzErkcnfxsRaU5oFgy2E5", "name": "Utopia Department of Motor Vehicles", "url": "https://dmv.utopia.example/", "image": "https://dmv.utopia.example/logo.png" }, "issuanceDate": "2023-11-15T10:00:00-07:00", "expirationDate": "2028-11-15T12:00:00-06:00", "name": "Utopia Driver's License", "image": "data:image/jpeg;base64,iVBORw0KGgoAAAANSUhEUg...kSuQmCC", "description": "A license granting driving privileges in Utopia.", "credentialSubject": { "id": "did:example:12347abcd", "type": "LicensedDriver", "driversLicense": { "type": "Iso18013DriversLicense", "document_number": "542426814", "family_name": "TURNER", "given_name": "SUSAN", "portrait": "data:image/jpeg;base64,/9j/4AAQSkZJR...RSClooooP/2Q==", "birth_date": "1998-08-28", "issue_date": "2023-01-15T10:00:00-07:00", "expiry_date": "2028-08-27T12:00:00-06:00", "issuing_country": "UA", "issuing_authority": "UADMV", "driving_privileges": [{ "codes": [{"code": "D"}], "vehicle_category_code": "D", "issue_date": "2019-01-01", "expiry_date": "2027-01-01" }, { "codes": [{"code": "C"}], "vehicle_category_code": "C", "issue_date": "2019-01-01", "expiry_date": "2017-01-01" }], "un_distinguishing_sign": "UTA", "aamva_aka_suffix": "1ST", "sex": 2, "aamva_family_name_truncation": "N", "aamva_given_name_truncation": "N" } } }
If you use full URLs, this is how developers access variables:
const familyName = vc["https://www.w3.org/2018/credentials#credentialSubject"]["https://w3id.org/vdl#license"]["https://w3id.org/vdl#familyName"];
If you use terms, this is how developers access variables:
const familyName = vc.credentialSubject.driversLicense.family_name;
Note that all of your database keys (for indexing and look up) also become full URLs if you use full URLs in the VC... which leads to considerable database bloat because you're repeating long URLs across your entire database, and all payloads, now. I don't know many developers that would prefer full URLs over using terms, and this is borne out through many developer APIs in existence today.
@David-Chadwick,
I appreciate the concrete text suggestion. I do think it would still be over-reaching a bit, though.
We're producing data model and securing mechanism specs to provide some core primitives to people so that they can figure out how to best put them together in the ecosystem. I think our job is only to document how the primitives work and then get out of the way for innovators to experiment in the ecosystem. For example, we should not be telling other people how they must go about designing their UIs. This latter bit is how I read your text on what to display to humans.
There are a number of things that are given if you read the JSON-LD spec (on which our specs rely) that I'm happy for us to point out again explicitly in our own specifications. I think that's helpful to implementers. For example, I do think it's good for us to say that context is used to map JSON keys to globally unambiguous URLs and that different contexts can map different JSON keys to the same or different URLs, enabling flexibility in a permissionless, decentralized ecosystem. I think it's a good idea to repeat that you must not consume terms defined by untrusted contexts and that you have the option to transform VCs to trusted contexts or to expand terms to full URLs.
But we shouldn't tell implementers what and how they must display things to users. This responsibility, and most others, fall to them. In my view, assuming that we know what is best for every circumstance not only exceeds our remit, but strips people of the freedom they need to employ their own intelligence and ingenuity.
On that point, I'll repeat that people are incubating interesting render methods that show neither URLs nor JSON keys to users, but provide more natural UI, such as skeuomorphic displays, within general purpose digital wallets. Reusable, pluggable software for these render methods can be written such that it will compact an incoming VC to a context for consumption, perform a JSON schema check, and display its contents via a visual template. Again, this is just one example -- and something that we shouldn't get in the way of here.
I finally had the chance to read this long thread. The problem statement is clear, though it seems there isn't yet consensus on the best way to move forward despite @msporny's enumeration 2 weeks ago.
Here is what I suggest as a concrete proposal:
@vocab
9 and 6 related to @vocab
, which are already in progress (see https://github.com/w3c/vc-data-model/pull/1520, https://github.com/w3c/vc-data-model/pull/1524).
Add normative language to the DI specification requiring implementers to rigorously validate @context
values. This should include:
a) Checking against a list of trusted contexts
b) Verifying the integrity of context contents using cryptographic hashes
c) Rejecting or transforming documents with untrusted contexts before processing
Develop an extension to DI proofs, including a cryptographic commitment to the @context
This allows verifiers to detect any tampering with the context after signing.
Update DI spec language to provide a transparent, step-by-step processing model for verifiers that emphasizes secure handling of contexts and transformation of documents to trusted forms before interpretation. We can highlight the potential attacks/vulnerabilities at each step and recommend mitigations.
Ensure the DI test suites include specific checks for these vulnerabilities, ensuring conformant implementations handle them correctly. This will be tough if we do not make strong normative statements about (1) and (2) above, but we can at least note conformance for implementers who have chosen to follow the guidance.
Given the many comments above, these proposals have nuance and tradeoffs, like privacy risks mentioned in being unable to swap out identical contexts, limiting decentralization/scalability, or reducing the flexibility of test/playground instances. The language we add should capture these nuances and ensure the guidance is crystal clear to minimize the security risks this issue was set out to address. It's up to the group to determine the strength of the language (SHOULD vs. MUST) given the aforementioned nuance/tradeoffs. Still, we should continue that discussion on the PRs that are open to addressing 1-3.
If you disagree with this proposal, please let me know a workable alternative.
@decentralgabe wrote:
Here is what I suggest as a concrete proposal:
- Adjustments to
@vocab
+1
- Enhance Context Validation (aligns with 4; this may already be present?)
+1
- Introduce Context Integrity Protection (aligns with 4; this is not present, and there are noted tradeoffs)
Strong -1, for all the reasons documented in this thread (it, debatably, doesn't properly address the issue, it makes certain use cases some of us are counting on impossible to achieve, it creates privacy harms, we'd be doing it at the last minute without much implementation feedback, etc.)
- Clarify the Processing Model
+1
- Improve Test Suites
+1 (noting that if we do this, we'll be wandering into "validation", "application layer", and "business rules" stuff, and that could kick off a whole new debate)
(edited to put backticks around @words
)
If you disagree with this proposal, please let me know a workable alternative.
@decentralgabe Additionally, I suggest adding some statements about how to process and validate DI-secured documents when the verifier does not know, understand, or trust all context entries but also does not care about the terms included in them. This ties back to the example with the extended types and respective context entries where the verifier only knows, understands, trusts and is only interested in the terms inferred by the general type defined by the general context entries. (cc @msporny @dlongley ).
Test suites design will be strongly coupled with what is defined in 1 and 3 (context validation + processing model).
The vcdm has a validation section, I could suggest adding a context validation element as part of that list.
I would recommend using the problem details feature and add directives in the algorithm section, instructing how to check against a known list of contexts, configurable by the verifier. I would see something along the lines of Verification software MUST enable developers to configure a list of trusted context urls. Such a list SHOULD contain well-known urls such as the ones depicted in the [data integrity specification](https://www.w3.org/TR/vc-data-integrity/#contexts-and-vocabularies) in addition to other contexts known to the verifier
.
To test this, there must be enough information to be asserted by a test client returned by the verification/validation process, where the problem details come into play. Currently, the vc-api is most likely the best tool to expose this verification response to a test client in the form of an HTTPS response body. I believe problems relating to unknown contexts are to be considered warnings
and shouldn't invalidate the signature but provide a warning to the controller/coordinator software to take action upon. The signature remains valid, but the context is unknown to the verifier therefore the data has to be handled with caution if not entirely rejected even if the signature is valid.
@awoie I believe this would apply to all validation of json-ld based Verifiable Credentials, irrelevant of which Securing Mechanism is used. The securing mechanism doesn't change the fact that the verifier should trust the context as this is part of validating the credential, not verifying it.
@awoie I believe this would apply to all validation of json-ld based Verifiable Credentials, irrelevant of which Securing Mechanism is used. The securing mechanism doesn't change the fact that the verifier should trust the context as this is part of validating the credential, not verifying it.
I believe that is true but it would be good that what needs to be done to achieve this is documented. It is also important for type-specific processing without general json-ld processing.
+1 for the need of improved documentation.
The only concern I have with type specific distinctions is the polyglot conundrum.
@PatStLouis wrote:
The only concern I have with type specific distinctions is the polyglot conundrum.
The basis of that post doesn't make sense... every JSON object is a polyglot based on what that post argues. JSON.parse() does one thing, and when an application developer uses their application to do anything more than JSON.parse(), they're operating as a polyglot. The second an application ignores a field because they don't understand it in a JSON document, they're operating as a polyglot. Even web browsers operate as polyglot systems -- take any CSS feature that's implemented in one browser but not another, or any feature that sits behind a flag, and you have a polyglot system. The VCDM has ONE data model with ONE syntax, that is JSON-LD... either you do type-specific processing on that JSON-LD document, or you do generalized JSON-LD processing on that document.
I will also warn folks that this thread is long enough without bringing other philosophical debates into this issue. Let's focus on @decentralgabe's proposal and work this issue to closure.
Maybe I misunderstood that post, or misunderstand what type specific means, but my point was that if someone issues and/or verifies a VC without treating it as json-ld (which it is) and then we need to define a different rule set about how to treat a VC as not json-ld, it adds a complexity that is unwarranted as VC were designed after json-ld. The VC spec shouldn't have to define a separate process to account for entities using VC as not json-ld.
The VC spec shouldn't have to define a separate process to account for entities using VC as not json-ld.
We had traditionally called this "JSON-only processing", but that term confused people and made them think that we were giving developers permission to interpret the document in a significantly different way from the result of a JSON-LD interpretation. That same confusion is where that post you linked to above was coming from (but also simultaneously says that VCDM v2.0 is no longer polyglot, and complains about that as well).
JSON-LD is designed so that you do not need to use a JSON-LD library to safely consume a JSON-LD document. For example, there are features (such as @context
and @protected
) and rules, such as those provided in Type-Specific Credential Processing, that instruct developers how to do that.
So, the VC spec isn't defining a separate process that gets separate results... it's defining a separate process that, as long as a few additional rules are followed, gets you to the same result as someone using a JSON-LD library.
Thank you for this information, I will do some reading. I retract my comments until I get a better understanding of this.
@decentralgabe wrote:
- Introduce Context Integrity Protection (aligns with 4; this is not present, and there are noted tradeoffs)
Just want to clearly express my support for this one, I believe this is the most important of the proposals to implement which actually mitigates the security vulnerabilities raised by this issue.
@msporny wrote:
Strong -1, for all the reasons documented in this thread (it, debatably, doesn't properly address the issue, it makes certain use cases some of us are counting on impossible to achieve, it creates privacy harms, we'd be doing it at the last minute without much implementation feedback, etc.)
Perhaps I'm missing something but I've not seen evidence of this not addressing the issue, with regard to usecases if you mean the ability to re compact a document against a new context, then this can be done when the @context
is integrity protected, it just happens post signature validation, which is a much safer time to do it, rather then blending it into the signature validation procedure. I also don't see what possible privacy harms come of this.
All —
Please review your comments above, and edit them to codefence all instances of @keyword
, including those in content within your comment which was quoted from someone else.
All you need to do is wrap those @keyword
instances in single backticks, like `@keyword`
.
Failure to do this has looped several GitHub users (e.g., @context
, @vocab
, @protected
, @base
, all of whom were tagged in the initial comment that opened this issue; I haven't reviewed the whole thread to make a list of all such) into this conversation without their permission nor request.
This is often shrugged off, as, what's the big deal about a few irrelevant notifications? Well, in this thread, it's more like 133 (and counting) irrelevant notifications. In most spheres where I travel, that would be classed as spam and/or harassment.
Please, let's be good gitizens.
The issue was discussed in a meeting on 2024-07-10
As an implementer of wallets, verifiers and issuers, and both DI and JWS libraries, I want to add my thoughts to this crucial discussion.
@contexts
SHOULD be digest hash protected, for both DI and JWS protected VCs. (I understand there may be some exotic use cases where the issuer might not want to sign over the context URLs and hashes, but I've also experienced first hand the needless confusion and wasted effort that results in NOT locking down contexts with hashes.) But I think those of us who write tutorials and implementation guides should STRONGLY recommend that the relatedResources
+ digest mechanism be used in VC design.
@context
s, should definitely be a general purpose mechanism (not even a Data Integrity specific one, just a general JSON mechanism). But I understand the practicalities of WG deadlines; fitting in to the VC DM is better than nothing (even though it would be even better to have it in a standalone mini-spec).relatedResource
mechanism does not protect contexts that are included in other contexts (like, for example, the v1 examples context, https://www.w3.org/2018/credentials/examples/v1 -- it included the ODRL context inside it). And so again, tutorials and VC implementation guides should STRONGLY recommend not embedding contexts like that.digestMultibase
mechanism is strictly better than digestSRI
, because it has two critical features: 1) it allows optional JCS canonicalization of JSON resources, whereas digestSRI does not. and 2) It can be base64url
encoded, and thus included in URLs, whereas digestSRI
is locked to (non-URL) base64
encoding.@protected: true
, BUT the @vocab
training wheels should be moved to the https://www.w3.org/ns/credentials/examples/v2 context (or rather, it's already there in the example context, so it should be removed from core VC DM 2.0 context).
https://www.w3.org/ns/credentials/examples/v2
(and its vocab) strikes the right balance of newbie-friendly, but also serves as a reminder that when it comes time for prod, the examples context should be removed (and undefined terms dealt with).Incidentally, digest hash protection of any resource, including @contexts, should definitely be a general purpose mechanism (not even a Data Integrity specific one, just a general JSON mechanism). But I understand the practicalities of WG deadlines; fitting in to the VC DM is better than nothing (even though it would be even better to have it in a standalone mini-spec).
FWIW, the issue of adding SRI to a @context
was on the agenda of the JSON-LD Working Group, but has been postponed for a future release, see https://github.com/w3c/json-ld-syntax/issues/108. I do not know what the plans are on actively reopening this issue in the JSON-LD WG (which is in maintenance mode now), but a future version of VC may well rely on a more general mechanism of a new version of JSON-LD. Which suggests that we should not define a new feature for VC but rely on, for now, the usage of relatedResource
.
Cc @BigBlueHat
@contexts SHOULD be digest hash protected, for both DI and JWS protected VCs.
@dmitrizagidulin could you clarify what you mean by "hash protected" here and how this protection would work? For example do you mean protected by the issuers signature like what has been suggested by @decentralgabe, myself and others? Or instead leaving the protection reliant on an out of band step similar to the current specification guidance and hence optional in practise when it comes to validating a DI proof.
@tplooker
@contexts SHOULD be digest hash protected, for both DI and JWS protected VCs.
@dmitrizagidulin could you clarify what you mean by "hash protected" here and how this protection would work? For example do you mean protected by the issuers signature like what has been suggested by @decentralgabe, myself and others? Or instead leaving the protection reliant on an out of band step similar to the current specification guidance and hence optional in practise when it comes to validating a DI proof.
At present, I think we should go with (and recommend to devs) the 'digest hashes signed over by issuer' like you and Gabe suggested. Specifically, the relatedResources
+ digest mechanism that's in the current VC DM 2.0, since those properties do get signed over (by both Data Integrity and JWS mechanisms).
Would I prefer adding the digest hashes to the URLs, inline? (So that links would look like this:
{
"@context": [ "https://www.w3.org/ns/credentials/v2#digest=12345"],
// ...
"evidence": [{
"id": "https://example.com/files/my-dissertation.pdf#digest=4567"
}]
}
Yes, I'd prefer that mechanism, in the ideal world (if we had time before CR to publish it as a standalone spec, AND if there was political will in the community to add support for this to JSON-LD libraries (or at least on the HTTP client level that those libraries use) in various programming languages). It's a somewhat better mechanism because a) It could be used in embedded contexts (where you add a link to another context inside a context, like examples/v1 did). and b) It's a useful general purpose JSON mechanism.
BUT, given the current landscape, it's more realistic to use relatedResource
& sign over it as usual, so that's fine. (and by "that's fine" I mean -- doing that is STILL drastically better than not using any sort of digest).
FWIW, the issue of adding SRI to a
@context
was on the agenda of the JSON-LD Working Group, but has been postponed for a future release, see w3c/json-ld-syntax#108. I do not know what the plans are on actively reopening this issue in the JSON-LD WG (which is in maintenance mode now), but a future version of VC may well rely on a more general mechanism of a new version of JSON-LD. Which suggests that we should not define a new feature for VC but rely on, for now, the usage ofrelatedResource
.
@iherman Oh, that's really cool! Thanks for the link. I really hope the future json-ld WG does get around to standardizing that mechanism!
I think I like what Dmitri's suggesting, but to clarify:
Having to use relatedResource
would be redundant for someone achieving this another way: content addressable storage, DID Linked resources, etc, and would cause unnecessary cruft.
Noting actions to raise PRs and issues taken on the last telecon.
@vocab
PRs have been raised for this item: https://github.com/w3c/vc-data-model/pull/1520, https://github.com/w3c/vc-data-model/pull/1524, and https://github.com/w3c/vc-data-model/pull/1524
Now being tracked in https://github.com/w3c/vc-data-model/issues/1529
A PR has been raised for this item: https://github.com/w3c/vc-data-model/pull/1535
A PR has been raised for this item: https://github.com/w3c/vc-data-model/pull/1537
Now being tracked in https://github.com/w3c/vc-data-integrity/issues/288
A PR has been raised for this item: https://github.com/w3c/vc-data-integrity/pull/291
Now being tracked in https://github.com/w3c/vc-data-model-2.0-test-suite/issues/88 and https://github.com/w3c/vc-data-model-2.0-test-suite/issues/88 with the expectation of PRs for discussion within a few weeks.
@dlongley The bit about display which you picked up on is peripheral to the main text. We can easily omit it so that the proposed text becomes
"DI signature verifiers MUST trust all the @context
definitions in the verifiable credential. Globally there are expected to be thousands of@context
definitions, and no implementation can feasibly trust all of these definitions. A few definitions will be widely trusted, such as the one published in the VC DM, or in another ISO or IETF standard or by a national or international authority. The remainder are likely to be only be trusted by a small subset of implementations. Consequently all implementations MUST discard all the non-trusted@context
definitions before processing the verifiable credential"
The issue was discussed in a meeting on 2024-07-31
The following issue outlines two significant security vulnerabilities in data integrity.
For convenience in reviewing the below content here is a google slides version outlining the same information.
At a high level summary both vulnerabilities exploit the "Transform Data" phase in data integrity in different ways, a process that is unique to cryptographic representation formats that involve processes such as canonicalisation/normalisation.
In effect both vulnerabilities allow a malicious party to swap the key and value of arbitrary attributes in a credential without the signature being invalidated. For example as the attached presentation shows with the worked examples, an attacker could swap their first and middle name and employment and over18 status without invalidating the issuers signature.
The first vulnerability is called the unprotected term redefinition vulnerability. In general this vulnerability exploits a design issue with JSON-LD where the term protection feature offered by the
@protected
keyword doesn't cover terms that are defined using the@vocab
and@base
keywords. This means any terms defined using@vocab
and@base
are vulnerable to term redefinition.The second vulnerability exploits the fact that a document signed with data integrity has critical portions of the document which are unsigned, namely the
@context
element of the JSON-LD document. The fact that the@context
element is unsigned in data integrity combined with the fact that it plays a critical part in the proof generation and proof verification procedure, is a critical flaw leaving data integrity documents open to many forms of manipulation that are not detectable through validating the issuers signature.Please see the attached presentation for resolutions to this issue we have explored.
In my opinion the only solution I see that will provide the most adequate protection against these forms of attacks is to fundamentally change the design of data integrity to integrity protect the
@context
element. I recognise this would be a significant change in design, however I do not see an alternative that would prevent variants of this attack continuing to appear over time.I'm also happy to present this analysis to the WG if required.
(edited to put backticks around
@words
)