w3c-ccg / community

COMMUNITY: W3C Credentials Community Group Community Repo
https://w3c-ccg.github.io/community
Other
42 stars 6 forks source link

[PROPOSED WORK ITEM] Test Suite for Verifiable Credentials v2.0 #241

Closed msporny closed 5 months ago

msporny commented 1 year ago

Abstract

The Verifiable Credentials v2.0 test suite provides a mechanism to test if an issuer or verifier is conformant with the Verifiable Credentials v2.0 data model.

Owners

@clehner @msporny (same organization, looking for a supporter from another organization)

Work Item Questions

  1. Explain what you are trying to do using no jargon or acronyms.

We are trying to create a mechanism that an implementer of the Verifiable Credentials v2.0 data model can use to determine if they are conformant with the Verifiable Credentials v2.0 specification.

  1. How is it done today, and what are the limits of the current practice?

At present, there is no test suite for the Verifiable Credentials v2.0 data model.

The Verifiable Credentials v1.1 data model required a number of things that made testing not as good as it could be, namely:

  1. What is new in your approach and why do you think it will be successful?

The current approach utilizes the Verifiable Credentials API to provide a unified interface for issuing and verifying a Verifiable Credential. This approach has the following advantages:

We believe this approach will be successful because it has already been used to demonstrate interoperability among 13+ implementers in the Jobs for the Future Plugfest. That is, we already have 13 implementers that can integrate with this test suite.

  1. How are you involving participants from multiple skill sets and global locations in this work item? (Skill sets: technical, design, product, marketing, anthropological, and UX. Global locations: the Americas, APAC, Europe, Middle East.)

This work item is highly technical in nature and executes upon the decisions made in the Verifiable Credentials v2.0 work, which has been demonstrated to be more broadly inclusive of different input from a variety of parties from around the world.

Given that this is a test suite, it's audience are implementers and not necessarily people involved in design, product, marketing, anthropological, and UX fields.

  1. What actions are you taking to make this work item accessible to a non-technical audience?

We are generating a human-readable report as a part of the test suite output with easy to read columns stating which implementations pass vs. fail certain tests. The report should be citable/referenceable by non-technical individuals to understand how interoperability in the ecosystem is doing from a week-to-week basis.

msporny commented 1 year ago

Here is some preliminary example output from the test suite below:

  Verifiable Credentials Data Model v2.0
      ✓ "Conforming processors MUST produce errors when nonconforming documents are consumed."
      ✓ "Verifiable credentials and verifiable presentations MUST include a @context property."
      ✓ Verifiable credentials and verifiable presentations: "The value of the @context property MUST be an ordered set where the first item is a URL with the value https://www.w3.org/ns/credentials/v2."
      ✓ @context: "Subsequent items in the array MUST express context information and be composed of any combination of URLs or objects."
      ✓ "All libraries or processors MUST ensure that the order of the values in the @context property is what is expected for the specific application."
      ✓ if present: "The id property MUST express an identifier that others are expected to use when expressing statements about a specific thing identified by that identifier."
      ✓ if present: "The id property MUST NOT have more than one value."
      ✓ if present: "The value of the id property MUST be a URL which MAY be dereferenced."
      ✓ "The value of the id property MUST be a single URL."
      ✓ "Verifiable credentials and verifiable presentations MUST have a type property."
      ✓ "The value of the type property MUST be, or map to (through interpretation of the @context property), one or more URLs."
      ✓ list: "objects that MUST have a type specified."
      ✓ "A verifiable credential MUST have a credentialSubject property."
      ✓ "The value of the credentialSubject property is defined as a set of objects that MUST contain one or more claims that are each related to a subject of the verifiable credential."
      ✓ "A verifiable credential MUST have an issuer property."
      ✓ "The value of the issuer property MUST be either a URL or an object containing an id property."
      ✓ "A credential MUST have an validFrom property."
      ✓ "The value of the validFrom property MUST be a string value of an [XMLSCHEMA11-2] combined date-time string representing the date and time the credential becomes valid, which could be a date and time in the future."
      ✓ "If present, the value of the validUntil property MUST be a string value of an [XMLSCHEMA11-2] combined date-time string representing the date and time the credential ceases to be valid, which could be a date and time in the past."
      ✓ "If present, the value of the credentialStatus property MUST include" id and type
      ✓ credentialStatus id property "MUST be a URL which MAY be dereferenced."
      ✓ "If present, the value of the verifiableCredential property MUST be constructed from one or more verifiable credentials, or of data derived from verifiable credentials in a cryptographically verifiable format."
OR13 commented 1 year ago

I don't believe the vc-api (a CCG work item) should be required to demonstrate conformance to the vc data model 2.0 (a W3C VCWG work item).

msporny commented 1 year ago

I don't believe the vc-api (a CCG work item) should be required to demonstrate conformance to the vc data model 2.0 (a W3C VCWG work item).

A couple of things to consider:

  1. This doesn't have to be the only way to test against the core data model, other implementers can create alternative mechanisms to test conformance.
  2. If we don't re-use just two endpoints of the VC API to drive this test suite (issue and verify), which already has 13+ implementers that integrate with it, a new bespoke mechanism will have to be proposed and created, generating even more work for an already overloaded WG with a solution that won't be reusable outside of this specification.

That said, if others would like to propose alternatives, the CCG Process allows for this. In the meantime, there are a large body of implementers that exist today that will be able to integrate with this test suite to demonstrate interoperability on the v2.0 VC data model.

mprorock commented 1 year ago

Quick clarification - some of the code is prefixed:

Copyright (c) 2022 Digital Bazaar, Inc. All rights reserved.

I think we will need this as a permissive license, etc to adopt.

msporny commented 1 year ago
Copyright (c) 2022 Digital Bazaar, Inc. All rights reserved.

I think we will need this as a permissive license, etc to adopt.

Yes, agreed, we'll have to fix that -- it's already under a permissive license, which is in the README.md:

https://github.com/digitalbazaar/vc-data-model-2-test-suite#license

Hopefully clarifies that the entirety of the test suite is already under a W3C docs and code license.

EDIT: I have now also created a LICENSE.md file to hopefully make that clear: https://github.com/digitalbazaar/vc-data-model-2-test-suite/blob/main/LICENSE.md

mprorock commented 1 year ago

I don't believe the vc-api (a CCG work item) should be required to demonstrate conformance to the vc data model 2.0 (a W3C VCWG work item).

Chair hat off: If we are going to use an API for any testing, I would strongly prefer postman/newman based tests

mprorock commented 1 year ago

per discussion on CCG call 2/28/23 Item should be brought directly to VCWG

dlongley commented 1 year ago

@OR13,

I don't believe the vc-api (a CCG work item) should be required to demonstrate conformance to the vc data model 2.0 (a W3C VCWG work item).

I completely agree -- but I don't think that's being proposed here. This is just one possible test suite that can be used to demonstrate implementation of the VCWG 2.0 data model. It's just reusing work that people are already using elsewhere to help demonstrate interop. Of course, everyone should be welcome to create their own test suites that people could also / alternatively implement against.

decentralgabe commented 1 year ago

supportive of this work item. agree that vc-api shouldn't be a hard requirement, but an option. also against companies needing to host infrastructure to participate.

msporny commented 1 year ago

supportive of this work item. agree that vc-api shouldn't be a hard requirement, but an option. also against companies needing to host infrastructure to participate.

I'll note that the following functions of the test suite are abstracted: issue(doc), verify(doc), prove(doc), verifyVp(doc)

https://github.com/digitalbazaar/vc-data-model-2-test-suite/blob/main/tests/10-vcdm2.js#L159-L162

... and the great thing about abstractions is you can use alternative mechanisms to issue/verify. So, if someone wants to do that through a command line interface to a docker instance (because they don't want to stand up infrastructure to run against the test suite), that can be done... if someone wants to do that via OIDC4 (because they prefer that protocol), they can... and if someone wants to do that via newman/postman, that's also possible (as long as people are willing to do the work).

mkhraisha commented 1 year ago

Supportive of this work item, and direction taken

wip-abramson commented 5 months ago

Update on this work item? I believe it is an active work item now right?

msporny commented 5 months ago

Update on this work item? I believe it is an active work item now right?

Yes, and already migrated to W3C: https://github.com/w3c/vc-data-model-2.0-test-suite

This issue should be closed (as completed).

wip-abramson commented 5 months ago

Great