Need to be able to issue multiple credentials following a specific template for agricultural use case

[mprorock]

We have a need to be able to issue multiple credentials following a certain object type at a single request. Frequently when an agricultural goods inspection occurs, multiple certificates (which share a parent type) need to be generated to be attached to a given shipment or pallet of goods.

What is the best way of going about this?

[OR13]

@mprorock In the version we used for the last plugfest: https://w3c-ccg.github.io/vc-http-api/versions/v0.0.1/

You would need to make a separate network request for each credential.

So you would setup your credential without a proof:

HTTP POST https://vc.transmute.world/v0.1.0/issue/credentials

{
  "credential": {
    "@context": [
      "https://www.w3.org/2018/credentials/v1",
      "https://www.w3.org/2018/credentials/examples/v1"
    ],
    "id": "http://example.gov/credentials/3732",
    "type": [
      "VerifiableCredential",
      "UniversityDegreeCredential"
    ],
    "issuer": "did:key:z6MkjRagNiMu91DduvCvgEsqLZDVzrJzFrwahc4tXLt9DoHd",
    "issuanceDate": "2020-03-10T04:24:12.164Z",
    "credentialSubject": {
      "id": "did:key:z6MkjRagNiMu91DduvCvgEsqLZDVzrJzFrwahc4tXLt9DoHd",
      "degree": {
        "type": "BachelorDegree",
        "name": "Bachelor of Science and Arts"
      }
    }
  },
  "options": {
    "proofPurpose": "assertionMethod",
    "verificationMethod": "did:key:z6MkjRagNiMu91DduvCvgEsqLZDVzrJzFrwahc4tXLt9DoHd#z6MkjRagNiMu91DduvCvgEsqLZDVzrJzFrwahc4tXLt9DoHd"
  }
}

example: https://vc.transmute.world/api/docs/static/index.html#/v0.1.0/post_v0_1_0_issue_credentials

A client would then need to construct a for loop, and loop over all values they wished to provide as input to the credential, and request each credential, one at a time, and await an http response.

the size of the http request would roughly match the size of the credential in json.

[mprorock]

One off requests seem highly problematic for our use case for a couple of reasons. When dealing with agricultural scenarios you may have no internet when conducting the inspection, and the data may be collected locally for each shipment and then batched out to issue when network is restored. Also, given frequent and multiple requests would be highly problematic if we have to wait for each one to resolve, vs just passing an array over of the inspection data that VCs need to be issued for, as there is often a moderate volume of VCs required at a processing facility for instance. A single bulk request is fine, but issue a ton of network connections on a limited internet gateway tends to be problematic.

[peacekeeper]

In the original version of the API there was a field called "templateReference" (see https://github.com/w3c-ccg/vc-http-api/blob/master/docs/versions/v0.0.0/vc-http-api.yml#L53), with this description:

Optional internal reference to a credential template. If this is present, it may be used to automatically pre-populate certain fields in the request body.

Note that this does not necessarily imply that we have to define the syntax of those references, or an actual template format. Those could be implementation-specific.

[mprorock]

@peacekeeper That templateReference might work, especially if we could pass in an array of rather than a single request. Overloading request body on rest APIs seems reasonably standard at this point, then the server can handle looping, parallelization, etc as desired by the implementation. Thoughts?

[peacekeeper]

@mprorock Hmm if it's an array, then what would be the elements in that array? Do you mean that an issued credential can be based on multiple templates?

[mprorock]

@peacekeeper we typically will see a case like the following: 1) Package arrives at a processing facility 2) sample from package is tested for phytosanitary requirements 3) sample from package is tested for contamination 4) package overall is inspected / treatments are verified 5) inspector enters data and VCs are issued In that case 3 VCs all share a common object type for the VC AgInspection which I was thinking of in relation to your link out to the templateReference

These types of inspections will happen on multiple containers and packages at the same time while harvest is going on, so we also have the case where a processing facility may pass us an array of inspections that differ in terms of which package or sample was inspected, but otherwise may have large amounts of common data, and all share a type (e.g. AgInspection or something like a PPQ Form 203 which itself contains an AgInspection object

[OR13]

sounds like the primary benefit of templateReference is reducing the size of a request to issue a large number of credentials at once.... the downside of templates as was discussed previously is that they are yet another data model.... I find that downside not a strong enough reason to continue to explore templates.

Similar to @mprorock , we would like to allow an issuer to do some setup, and then issue large volumes of credentials very quickly, with low http overhead.

an issuer might create 3 templates, and then provide a single post request which referenced the templates and ONLY provided the values needed for each credential (stripping JSON-LD bloat from the inbound http api)....

There would still need to be max size restriction on such as api.

IIRC one of the major objects to the template approach was the initial aesthetic.... I recall feeling not super comfortable with it. @peacekeeper @mprorock perhaps we can align around a template language for JSON as a first step?

There are 2 categories of template languages.....

1. ones that support logic
2. ones that don't

the ones with logic are more powerful and more dangerous, but for example they could handle the case of:

"I want an issuer to be an IRI vs I want an issuer to be a JSON-LD object"....

This is a case we care about, we would love to be able to support both cases with a single template.

However, that increased complexity is maybe not a good idea for a v0 system.....

We can easily avoid that, by having all values needed by a template be required.... for example, a template might look like:

{
"issuer": ${issuer.id}
}

according to https://tools.ietf.org/id/draft-jonas-json-template-language-01.html

and its template vars (or arguments), can be described with JSON Schema....

{
"type": "object",
"properties":{
"issuer":{
"type":"object",
"properties":{
"id": {
"type": "string"
}
}
}
}
}

The process of issuing via a template is then as follows:

1. create the template (rejecting templates that are malformed), included a json schema for validating arguments
2. reference the template by id, and provide an array of objects that are valid according to the schema.
3. on the issuer api construct the credential payload input for each credential by applying the template args to the template
4. sign each credential

One of the challenges in doing batches of credentials like this is support for multiple signing keys , "verification options".... I would propose that all credentials issued from a batch MUST use the same options, this makes things simple.

[mprorock]

@OR13 yes - I think shared use case on the issuing large volumes of credentials with a minimum of HTTPs requests. I think a JSON template spec might be a good first step, then we could mock up an example or two of what it could look like.

[peacekeeper]

Since @OR13 just wrote some wonderful example code in https://github.com/w3c-ccg/vc-http-api/issues/41#issuecomment-771041038 that explores a potential subjectReference field, let me quickly try explain how I originally imagined the templateReference field (code copied and modified from that other comment):

const credentialTemplate = await getCredentialTemplateByReference(templateReference);
const credentialCreatedFromTemplate = {
...credentialTemplate,
...credential,
}
const vc = issue(credentialCreatedFromTemplate, {options: suite, documentLoader})

Defining a template language wasn't actually part of the idea. Also, personally I never had a strong use case for this, it was just an idea at that time; but maybe it can be useful in some way for batch issuing.

[OR13]

I would prefer to have a template language and do this instead:

const credentialTemplate = await getCredentialTemplateByReference(templateReference);

const credentialCreatedFromTemplate = apply(credentialTemplate:string, templateArguments:object)
const vc = issue(credentialCreatedFromTemplate, {options: suite, documentLoader})

then the template language can be anything and the arguments can be whatever you would normally pass to the language.

The spread or Object.assign approach assumes JSON and is not a really good solution for satisfying a complex template.

[peacekeeper]

I would prefer to have a template language and do this instead:

This also looks useful! Maybe something like this exists already in the JSON-LD universe and can be re-used?

[OR13]

• https://tools.ietf.org/id/draft-jonas-json-template-language-01.html
• https://github.com/pallets/jinja
• https://github.com/handlebars-lang/handlebars.js

I don't think we should care about JSON-LD as much as we care about JSON for templating, especially if we want to use the same structure for vc-jwt.

by taking the interface as:

const credential = apply(template, args);

We allow for any template language, but i think we still may need to formalize some structure for the args, like JSON Schema...

Then the batch issuance process can be made more efficient by the following:

issueBatch = (templateRef, args) => {
 template = getTemplateByRef(templateRef); 
 credentials = args.map((arg)=> apply(template, arg)); 
 verifiableCredentials = credentials.map((credential)=> issue(credential, options));
 return verifiableCredentials
}

where the input is much smaller since it need only include a string to identify the template, and an array of arg objects to be used to inflate the credential just prior to signing.

[msporny]

We discussed this on the 2021-11-16 call. This is related to role of templates in batches of templates. Community consensus should be single request, no batch issuance endpoint. Batching will happen elsewhere. We could potentially close this issue -- we discussed this, people don't want to do batch endpoints with HTTP (good reasons, horizontal scaling, what happens if one credential, atomicity all come up). Batches of credentials that are revocable were a concern, batching is complicated and is outside of capabilities of this particular API. There is an argument for this being out of scope because of templating being out of scope.

There were no objections to closing this issue as it is currently out of scope.