Merge Issuer and Verifier API Specifications

[OR13]

https://github.com/w3c-ccg/vc-issuer-http-api https://github.com/w3c-ccg/vc-verifier-http-api

Managing issues / comments / PRs / versions for 2 repos, is twice as hard, will take 4 times as long...

I wouldn't be opposed to splitting the spec once its finalized, but I suspect we are creating pain for everyone with our current approach...

Its likely that many implementers will support both APIs, and use a base URL like;

https://example.com/instance/123/issuer/credentials https://example.com/instance/123/verifier/credentials

We can have:

https://example.com/instance/123/issuer/docs https://example.com/instance/123/verifier/docs

Or:

https://example.com/instance/123/docs

Other implementers might choose to add additional endpoints for convenience along side the required and CCG specified ones... how should this be handled?

Will every server need to create:

https://example.com/instance/123/normal-api/docs https://example.com/instance/123/issuer/docs ?

I would prefer that I be able to implement a single api, and attache the issuer / verifier routes to it, along side my existing routes, so I have 1 swagger, that contains issuer endpoints, verifier endpoints, and proprietary ones....

https://example.com/instance/123/docs => Swagger HTML for whole instance https://example.com/instance/123/docs.json => Swagger JSON for whole instance

[dhh1128]

I would prefer to have "issuer" and "verifier" in the URLs. I agree that they may commonly be implemented together, but I believe they might evolve a little bit independently, version-wise, and I want to be able to upgrade one without upgrading the other.

I don't have a strong opinion about docs.

[OR13]

Per my PR here: https://github.com/w3c-ccg/vc-issuer-http-api/pull/25

curl -s -X POST https://example.com/issuer/credentials \
-H 'Content-type: application/json' \
-H 'VC-API-Version: 0.0.1' \
-d @./payload.json  

curl -s -X POST https://example.com/verify/credentials \
-H 'Content-type: application/json' \
-H 'VC-API-Version: 0.0.2' \
-d @./payload.json  

would both be valid, even if the latest api swagger spec were 0.0.5... (assuming the endpoints were the same)

The docs for this would be:

https://example.com/docs => html
https://example.com/docs/json => json

If you wanted to prefix the paths, you could choose to do so:

https://example.com/vc-api/docs
https://example.com/vc-api/docs/json
https://example.com/vc-api/issuer/credentials
https://example.com/vc-api/verifier/credentials
https://example.com/proprietary/endpoint

[OR13]

Here is an example of what I am talking about: https://vc.transmute.world/api/docs/static/index.html

Note that I am defining the well known uri used by did web in the same swagger definition... why? because its used as the issuer did...

Should I have to host a whole separate swagger defintion to do that?

[msporny]

-1 for doing this in the next few weeks, keep them separate for now, we're dealing with enough churn... you've got some deployment ideas baked into your argumentation above that don't apply to what Digital Bazaar is doing... so, some of those arguments are falling flat for us. I agree that we may merge them in time, but let's let these two specs evolve separately and then make a decision to merge them if/when things calm down a bit.

[OR13]

@msporny can you post the swagger documentation for what you are doing so I can show you what I mean?

What is wrong with:

• https://example.com/baseBath/docs
• https://example.com/baseBath/docs/json
• https://example.com/baseBath/issuer/credentials
• https://example.com/baseBath/verifier/credentials

?

[msporny]

I think I see what you're saying now... if we're not miscommunicating, I'm fine with it (in general)... I just didn't want to put yet another decision on everyone's plates before Friday... which is why I -1'd it for the next few weeks.

In other words, I think we should do what you're saying, but not now... later, once some of the dust has settled.

My comment wrt. "you've got some deployment ideas baked into your argumentation above" was merely referring to an assumption that you might have that there is always an issuer API alongside a verifier API. Or that you publish your documentation using Swagger. Our systems don't make either of those assumptions.

[OR13]

Yes, I agree a system might not publish both sets of endpoints, or use swagger at all.. The spec could use a single swagger definition to define both sets of endpoints, and it would help us communicate how the API is meant to be used.

[peacekeeper]

I also prefer to keep them separate for now.

[msporny]

Looks like #14 is blocked because of this issue, so let's get this resolved quickly. Let's discuss on Tuesday's call... I'm fine w/ the spec merge (all commits, all issues, etc.), the sooner the better.

[OR13]

I was able to get this working the way I wish we had things...

https://github.com/OR13/respec-swagger https://or13.github.io/respec-swagger/

1 spec file (index.html) 1 api.yml 1 README 1 repo....

[OR13]

I'm happy to take point on refactoring the issuer api to this format / or the verifier api... or we can leave them separate and create a new ccg repo for the combined work, and not be disrupted, until its ready to have both pasted in...

[OR13]

https://stackoverflow.com/questions/1683531/how-to-import-existing-git-repository-into-another

https://medium.com/altcampus/how-to-merge-two-or-multiple-git-repositories-into-one-9f8a5209913f

I'm not the best with git, but I am happy to give this a shot.

[OR13]

I created https://github.com/w3c-ccg/vc-http-api, and followed https://medium.com/altcampus/how-to-merge-two-or-multiple-git-repositories-into-one-9f8a5209913f

This should work, but would be a lot easier if we temporarily moved all files in both repos into an "issuer" + "verifier" folder before attempting the merge.

Issues can easily be moved, but the current PRs are obviously going to confound our ability to merge these things, if we can get consensus to do that.

[msporny]

I was able to get this working the way I wish we had things...

+1 to the plan... note that example you have throws ReSpec errors every couple of reloads, which is the same problem I had here:

https://github.com/w3c-ccg/vc-verifier-http-api/pull/14

[llorllale]

Its likely that many implementers will support both APIs, and use a base URL like;

https://example.com/instance/123/issuer/credentials https://example.com/instance/123/verifier/credentials

There's an obvious case for the issuer to also be a verifier, but I don't see it so strongly the other way around. Can you elaborate?

Are we assuming issuers make up a certain percentage threshold of the VC ecosystem? Are we assuming a certain percentage of verifiers will also be issuers?

[OR13]

I'm confused... isn't the other way around that the verifier is also an issuer?

For example:

Lets say an issuer will only create VPs if they have some valid VPs from the subject already...

You show up at the DMV, and they verify your Birth Certificate Credential and Social Security Certificate Credential, then they issue your Drivers License Credential.

https://dmv-api.example.com/verifier/presentations
https://dmv-api.example.com/issuer/credentials

Are both called by the software that supports this case? or does the DMV use 2 separate APIs to support their VC - DID integration?

What if the system first wants to do mutual auth over CHAPI before exchanging presentations, does it use another API to support that?

[OR13]

Another example would be that you will get a receipt which is signed by the verifier (acting as an issuer)... saying that your data was received, and verified, and will be used according to ToS... similar to kantara consent receipts... https://kantarainitiative.org/wp-content/uploads/2014/10/Kantara-Consent-Receipt-Presentation.pdf

[dhh1128]

The terminology is causing problems for me here. Let me say how I understand the terminology (which I think and hope is how everybody else does, too) just as an alignment check. Then I'll reason about the github issue in general.

Words like "Issuer", "Verifier", "Holder" and "Prover" label the role that a given party is playing in a given interaction -- never an abstract role in the ecosystem that is held without an interaction context. The DMV (a concrete entity) can totally be an Issuer in one interaction, and a Verifier in another -- but when they're acting as an Issuer, the thing they're doing is always "issuing," never "proving" or "verifying." And when they act as a Verifier, the thing they're doing is always "verifying," never "issuing" (even if they're in the middle of issuing, somewhere upstream). And when they'e not in an interaction, they're neither an Issuer nor a Verifier.

Thus, the phrase "an issuer will only create VPs" makes no sense. By definition, Issuers (the party that, in an issuance interaction, generates new credentials to give to a holder) never create VPs; indeed, the issuer role is not involved with the "present" verb, ever, period. Only Holders who want to become Provers create VPs.

What I think we mean by this phrase is that the party who was in the middle of issuing in one interaction suddenly switches to being a Prover that generates VPs in another interaction (possibly with the same complementing party, who has also changed roles). That makes sense. But when they do that, they enter a new interaction in which they are not analyzed as an Issuer anymore.

Now I can use terminology carefully to comment about the github issue.

An institution is likely to implement both an issuance API and a verification API. We hope they implement them to conform with the standards we are describing here. All issuers should be able to play the verifier role as well, and all verifiers should be able to play the issuer role as well, by the time we have a mature ecosystem. Not all institutions will want to issue high-quality, reusable, multicontext VCs like a driver's license, but almost all institutions will probably see a need to issue receipts and similar pieces of more narrowly useful data, sooner or later. And almost all institutions will need to verify stuff.

No issuers ever "issue" or "present" VPs. Only Holders who want to be Provers do that (and their verb is always "present", never "issue"). Of course institutions could also be Holders who want to be provers. But the scope of the issuance API should not include VPs, period. The scope of the verification API should, on the other hand.

Describing the issuance API and the verification API with similar conventions, and possibly in a single repo, seems rational to me. Not compelling, but acceptable. (Orie can tell horror stories about the pain of fragmentation; I can tell horror stories about the pain of monoliths. The disadvantages of both are avoidable, and the advantages of each strategy are replicable in the other, too. So it's a matter of preference. Since Orie is doing a lot of legwork here, I'm fine with his preference carrying the most weight.)

Describing both APIs in the same Swagger file seems a little bit iffier to me, but I'm willing to go along with that as well, deferring to Orie.

What I'm not okay with is a strategy that requires these APIs to be versioned and released together. I want to be able to release one without the other. As long as I have that, and as long as we don't pollute issuance with presentations or verification with issuance, I'm cheerful about the rest of the approach.

[OR13]

@dhh1128 thanks, you are right to object to imprecise term use. please continue to do so :)

One thing worth noting is that we have already agreed to supporting at least api versioning using http headers.

Another is that while I'm suggesting merging the specification into 1 repo for the sake of development and sanity, I am NOT suggesting that any given API must implement both...

Some APIs will implement both, and might use swagger to express a set of interfaces that looks almost exactly like this spec... Others might only implement one or the other.

Since it was mentioned, its worth sharing: https://martinfowler.com/microservices/

and https://www.infoq.com/news/2019/04/monolith-vs-microservices/

In a nutshell, I'm arguing that we don't understand the HTTP API at all and that its too early to "microservice-ize it"... because we don't yet have agreement on really anything...

I prefer to build a monolith, analyze it, then split it into microservices, as scaling requirements demand that... not split things up into modules before we have any idea what the modules will be doing or consensus on the basics of api design.

Scaling or optimizing too early is almost always a mistake... one which I believe we are making right now, by keeping these API specs in separate repos.

I want to call out this quote:

An institution is likely to implement both an issuance API and a verification API. We hope they implement them to conform with the standards we are describing here. All issuers should be able to play the verifier role as well, and all verifiers should be able to play the issuer role as well, by the time we have a mature ecosystem. Not all institutions will want to issue high-quality, reusable, multicontext VCs like a driver's license, but almost all institutions will probably see a need to issue receipts and similar pieces of more narrowly useful data, sooner or later. And almost all institutions will need to verify stuff.

I think this gets to the heart of making these APIs understandable for institutions, and its something which is not working well for us right now by trying to use swagger out of the gate.... we should be constructing english sentences, like this... before attempting to convert "stories" into "http endpoints".

[dhh1128]

Largely, +1 to everything above.

It's worth noting that we (Orie and I) may still be talking about different issues with respect to versioning and releasing. His comments above seem to focus on how a particular implementation of the specs does its versioning and releasing; certainly the comment about communicating versioning in HTTP headers is consistent with that.

I'm more worried about versioning and releasing the specs themselves. I want to be able to have a spec for version 1.1 of issuing that is independent of the spec for version 1.0 or 1.2 of the spec for verifying. I do not want to be forced to release one spec just because I'm releasing the other.

I prefer to build a monolith, analyze it, then split it into microservices, as scaling requirements demand that... not split things up into modules before we have any idea what the modules will be doing or consensus on the basics of api design. Scaling or optimizing too early is almost always a mistake... one which I believe we are making right now, by keeping these API specs in separate repos.

It feels to me like this comment mixes concerns about implementation and specification. Are we doing both in this effort?

I claim that it's pretty cut-and-dried that issuance and verification are independent business problems. It might make sense to implement their solution together, but I don't think there's any question that they need independent specifications. So that's the tension: if we're doing impl, everything Orie said makes sense. But if we're writing specifications, not so much.

This is NOT me requesting a change to Orie's plan to merge repos. I'm just trying to force us to decide whether the focus is on impl or spec, and then not argue for processes using justifications that apply to a non-focus.

[OR13]

@dhh1128 any frustration that is visible in what follows is not directed at you or anyone else, i'm just very passionate about API design :)

The spec is not about business problems, the spec is about a data model and interfaces.

It is the defintion of the interfaces: noun / verb / vc / vp, object schemas, etc... that we don't have clear alignment on... having 2 modules / 2 repos for the 2 specs is contributing to the conflation of business needs with data model architecture....

Why should we have separate versions of the issuer and verifier api spec?

Do we have separate versions of the vc data model spec for VCs and VPs or issuing and verifying?

Why are we splitting something that was combined in the vc-data-model, when trying to make an HTTP API for the vc-data-model?

I want an HTTP API Spec for:

Creating & Verifying a VC / VP with support for proofs in JWT / JSON-LD / ZKP

I don't want to have 2 sets of REST endpoints which we decided to separate based on roles that someone might take when accessing them, because as was pointed out before...

An individual or institution might act as both an issuer and a verifier... so why should they need to review 2 repos, 2 specs, and implement support for 2 sets of APIs...

I don't use all the endpoints defined here:

https://developer.github.com/v3/

And I don't need to browse 2 sets of api definitions... for

https://developer.github.com/organizations/v3/ https://developer.github.com/issues/v3/

etc... we are splitting things up in a way that makes contribution and comprehension more difficult than the vc-data-model did when it implemented a single spec for both VC/ VP and issuing and verifying.

[dhh1128]

Wow, we are pretty misaligned.

The spec is not about business problems, the spec is about a data model and interfaces.

"Data model" as used in W3C specs that I know refers to the different data definitions and relationships in the abstract, and very much NOT to the format that data takes when it's realized in a digital file, an HTTP payload, etc. Certainly this is true about the VC Data Model spec that provides background for our effort here. Thus, what we are working on here is very much NOT a data model; rather, we are working on a data format (for payloads), which is quite different. And we are working on an API, which is about interactions around use cases, which = business problems. If we're not about use cases/business problems, what are we doing and why do we think it matters?

Why should we have separate versions of the issuer and verifier api spec?

Because, in a radical difference to the VC Data Model spec, the specs we're talking about here are about interactions. This is what makes them worthy of an API instead of just a data model. And the issuance interaction and the verification interaction are radically different. They involve different parties (or the same parties with different roles and responsibilities in the two contexts), with different goals, at different times.

Do we have separate versions of the vc data model spec for VCs and VPs or issuing and verifying?

Having one spec for automobile parts, no matter which automobile you're talking about, makes sense. Having one spec that covers both safety inspecting cars and selling cars (both of which depend upon the same auto parts, but that care about them for different reasons, to different degrees, with respect to different interaction partners) doesn't make as much sense, because they're independent interactions, not just inert data. The independence of the interactions is an important property. Note that I'm not claiming it's crazy to have one interface for both inspecting and selling cars; I'm just claiming that combining is by no means a slam dunk, and you're using a false parallel to reason from the VC data model spec.

Why are we splitting something that was combined in the vc-data-model, when trying to make an HTTP API for the vc-data-model?

We're NOT trying to make an HTTP API for the vc-data-model. We're trying to make two APIs that only have an indirect relationship to the VC data model. One is an internal API that issuers will use for their own internal processes, and another is an internal API that verifiers will use for their own internal processes. We're explicitly excluding from consideration the problem of interacting with another independent identity around either the issuance or the verification workflow.

I want an HTTP API Spec for: Creating & Verifying a VC / VP with support for proofs in JWT / JSON-LD / ZKP

I don't. I want that to be two different APIs that exhibit useful commonalities.

I am expecting vulnerabilities to be reported -- lots of them, for years to come. Credential fraud is a major issue, not a minor afterthought. When a vuln against the issuer API is reported, I want to quickly patch the issuance spec without having to re-release verification. And vice versa. And I want impls to be able to declare their conformance to a patch level for one independent of the other.

we are splitting things up in a way that makes contribution and comprehension more difficult than the vc-data-model did when it implemented a single spec for both VC/ VP and issuing and verifying.

I'll keep saying this. The VC Data Model spec IS NOT a spec for issuing and verifying. It's a spec for a data model. Data models are inert. There is no verb in them. There is no interaction recipe. They just say, "Here are the types of data you have to model, and how they should be defined, and how they relate." Reasoning from one of those to one of what we have here is a false equivalence.

BTW, when you make lists like "JWT / JSON-LD / ZKP", I think there's another false equivalence at play. I get why it's convenient (and reasonably accurate, if we squint) shorthand, but it may be worth commenting on that a bit. ZKPs are not a data format at all, but they have semantic ramifications on the data model, so they show up in the data model spec as a notable variation (to the model) in some places. There is more than one ZKP-oriented data format, but they don't show up as formats in the spec, only as a category of data model variation. JWTs are mostly a data format (a perfectly reasonable one); to the extent that they're format- rather than model-related, they don't need to show up as a unique variation in the spec, and they're still perfectly canonical. And JSON-LD is both a format and a data model construct; all data formats for VCs should exhibit certain JSON-LD-isms.

ZKPs are fairly easy to reconcile into the data model, but they imply a different interaction pattern. Most importantly, you can't prove something and know you've achieved zero knowledge unless/until you know what the proof criteria are. Thus, you can never share a ZKP until you've been challenged for one. There are other interaction consequences, too. Thus, fitting them into a spec that is about interaction (API) is going to require more flexibility than just adding a few fields to a data model. And different ZKP styles may have different interaction patterns vis-a-vis one another, too. A lot of the conversation we're having over on the CCG mailing list about ZKPs is colored (for me) by this background.

[dlongley]

+1 for different specs for issuing and verifying; they are for different interactions and need to be able to evolve (and get patched) independently from one another. I don't mind if these two specs live in the same repo or not -- that's a management issue. But they should be different, independent specification documents.

Regarding creating signed presentations -- an HTTP API for that would be independent from the issuing API and the verifying APIs. One would not request a VP to be "issued" via the issuer API; that's the wrong terminology. VCs are "issued" and then they (or other credentials that are derived from those VCs, as in the ZKP case) are "presented" via VPs. The only interaction I'm aware of that requires a signed presentation in the VC ecosystem is where the holder plays the role of the creator and signer of that presentation. So if the community saw a need for a standard HTTP API for that, it would entail some kind of holder API and it would likely be implemented by wallet/agent software. But there's no community effort on even determining a need for that right now. The focus is on issuing and verifying.

[OR13]

Are we sure we want to use the term "signer" for VPs, doesn't that bias against ZKPs / other proof formats?

What mechanism other than an HTTP API would did:web use with CHAPI to handle DID Auth (missing expected functionality)?

How will a user generate a VP that would be acceptable to the verification endpoint without an HTTP API to create it (missing expected functionality)?

How would I get a VP with proof generated from a hardware backed enclave on a remote server without something like HTTP (missing security support)?

I'm not likely to be convinced that sending a developer 2 sets of respec + swagger is better than sending them one... especially if both sets don't contain expected functionality needed to interact with each other.

We have an opportunity to clearly and simply support the vc-data-model using HTTP... I believe we are harming the adopting of the VC Data Model with the current approach.

[dhh1128]

Are we sure we want to use the term "signer" for VPs, doesn't that bias against ZKPs / other proof formats?

I thought the term was "Prover," and that this terminology has long since been canonized.

How will a user generate a VP that would be acceptable to the verification endpoint without an HTTP API to create it (missing expected functionality)?

See Aries RFC 0037, which works with HTTP and other transports. What's described is castable as an HTTP API, but is deliberately more generic. That's what I want -- something that allows HTTP as a super-important transport, but something that doesn't tell the world that the standard way to do it is HTTP, and anything else is off the table.

What I feel like you're arguing, Orie, is the importance of HTTP. We're in violent agreement there. But from that axiom, you go to: "we need a standard that's an HTTP API" -- and where I go is, "we need a standard where an HTTP API is well supported." Those are very different conclusions, but I am not seeming to succeed on getting you to engage on their difference. What justification can you offer -- NOT for the assertion that HTTP is important, but for the assertion that our standard should disallow anything else (or diverge from anything else) by defining everything in purely HTTP terms?

How would I get a VP with proof generated from a hardware backed enclave on a remote server without something like HTTP (missing security support)?

See Aries RFC 0037, which has no problem with this use case.

I'm not likely to be convinced that sending a developer 2 sets of respec + swagger is better than sending them one... especially if both sets don't contain expected functionality needed to interact with each other.

Okay. We can agree to differ. But note that I'm not actually hung up on giving devs 2 sets of respec + swagger; I'm hung up on independently releasing and versioning specs for 2 independent interactions. If you want to do 2 releases with 1 file, and pay the cost of separating the monolith at release and versioning time, that's okay by me.

[kimdhamilton]

done