Add representation-specific entry map in Production and Consumption.

[msporny]

This PR attempts to address @peacekeeper's long standing concern wrt. production and consumption and representation-specific entries by using the proposal for Option 3.3 in issue https://github.com/w3c/did-core/issues/664#issuecomment-780647353. This is also an attempt to get PRs #596 and #597 unstuck.

Specifically, it defines the input and return parameters for conforming producers and consumers and splits the representation-specific entries into their own map that can be serialized and returned along with the DID Document data model. It adds a requirement that all representations detect known representation-specific entries, which is required to cleanly create the two classes of information. The change is substantive.

---

Preview | Diff

[msporny]

I don't understand why we define abstract function for resolution, and then not for production and consumption.

With my Editor's hat firmly on: We never should have defined abstract functions for resolution using our own bespoke abstract function language... I cannot underscore how terrible the half-measure that is the Resolution section is -- we're bending over backwards and doing really strange stuff there to half-define something. We should have either just kept all of it to a testable algorithm in prose, like https://www.w3.org/TR/json-ld11-api/#expansion-algorithm does... or actually defined a concrete set of functions using Javascript/Typescript/WebIDL. What we have is a weird half-measure and I'm reluctant to double-down on that mistake.

I did start to write what you suggested up initially, and then decided against it because...

representation-options

We have no way to realize this parameter in the way the Resolution section is written (which again, is strange and bespoke and isn't clearly specified for anything that is not INFRA->JSON... like, you know, CBOR). None of the data types in the data model or INFRA support it. Consider the use case of injecting encoded CBOR values into the root CBOR map... integers as keys -- can't do it... no concrete representation in INFRA for CBOR. So the best I could do is use prose to say that you can take other options and those options can be used to affect production. We're not testing MAY statements... that's the only way I could get to making representation-options a possibility.

It's not that it's impossible to write the spec up with bespoke abstract function language... but it makes it worse, IMHO, not better. All that said, I'll draft a PR that does the abstract function thing (even though I'm totally against the concept... what we have now is testable and flexible enough to support representation-options). If I write it up like we did for Resolution, representation-options are still not going to be a thing because we have no way of expressing them.

[OR13]

We should have either just kept all of it to a testable algorithm in prose, like https://www.w3.org/TR/json-ld11-api/#expansion-algorithm does... or actually defined a concrete set of functions using Javascript/Typescript/WebIDL.

I agree... it's not too late to fix this ambiguity. I personally find function definitions far more useful than english sentences....

didDocument =  « 
didDocumentEntries: INFRA<map>, 
didDocumentRepresentationEntries: INFRA<map> 
»

resolve(
did: INFRA<string>, 
resolutionOptions: INFRA<map>
) →
   « 
   didResolutionMetadata: INFRA<map>, 
   didDocument: INFRA<map>, 
   didDocumentMetadata: INFRA<map> 
   »

resolveRepresentation(
did: INFRA<string>, 
resolutionOptions: INFRA<map>
) →
   « 
   didResolutionMetadata: INFRA<map>, 
   didDocumentStream: INFRA<ByteStream>, 
   didDocumentMetadata: INFRA<map> 
   »

consume(
contentType: INFRA<string>, 
didDocumentStream: INFRA<ByteStream>
) →
  « 
  didDocumentEntries: INFRA<map>, 
  representationEntries: INFRA<map> 
  »

produce(
contentType: INFRA<string>, 
didDocumentEntries: INFRA<map>, 
didDocumentRepresentationEntries: INFRA<map>
) →
  « 
  didDocumentStream: INFRA<ByteStream> 
  »

[msporny]

@OR13 wrote:

I personally find function definitions far more useful than english sentences....

Sure, I do as well, and W3C has a formal language in which to do that... it's called WebIDL... which people objected to using.

Instead, we've got a interface definition language that we invented on top of INFRA -- totally bespoke, no one understands it except for us... and we're using that to define the interfaces in the specification, which BTW, we're totally not chartered to do. The WG decided to look the other way wrt. the Resolution section on that particular point.

That's what I mean by doubling down. I agree that it's valuable, but skirting dangerously close to going outside of our charter and speaking in tongues (bespoke IDL) is the thing that's concerning to me.

[msporny]

This PR is blocked waiting on a review from: @peacekeeper and @jricher

[msporny]

@peacekeeper wrote:

We can still add examples for WebIDL (#587) and TypeScript (#591), I think that would be useful

WebIDL is normative -- it's not for examples. When one writes the WebIDL, you can say: "Do this \, or something that is functionally equivalent." It's the "something that is functionally equivalent" that gives us the ability to handwave over the other valid options. In other words, it's easier to look at a WebIDL interface and then see if someone has done effectively the same thing in another programming environment. The alternative is our bespoke INFRA-based IDL language that we've invented... which requires human intervention and tribal knowledge to read the tea leaves.

as long as it's clear that they are just examples, not the authoritative definitions.

I don't understand this... what is the authoritative definition, then? Where is it?

My preference at the Amsterdam F2F was also to not include DID Resolution in DID Core, but since then we've operated under the assumption that we want to define the "contract" (without its implementation) of DID Resolution (and DID URL Dereferencing).

Then I think it's time to revisit that decision given new information. As far as I can tell, there are a number of people in the WG that are 1) asking for this without grasping the ramifications, 2) asserting that we shouldn't set the bar that high, 3) feel like our bespoke IDL is appropriate, or 4) feel like the Resolution contract is fundamental to the specification.

With respect to 1 -- the people that have to write tests on this section are concerned and we should hear them out.

With respect to 2 -- yes, we can lower the bar; that leads to interoperability risks.

With respect to 3 -- our bespoke IDL requires tribal knowledge and human intervention to reason through, which is problematic.

With respect to 4 -- there are multiple others parts in our specification where we point out to sections of other specifications -- for example, URI, JSON, CBOR, INFRA, HASHLINK -- we can point out to an external RESOLUTION specification. I have tried to be very careful about the references to the RESOLUTION section such that they could be moved out if need be... that section is not intertwined with the specification and could easily be moved out if we needed to do so. Both specifications would be strengthened by doing so -- RESOLUTION could be put on a path to become entirely normative instead of being just a contract. This is what's going to happen with that specification in time, anyway.

Like @peacekeeper, my preference would be to publish Resolution in a separate NOTE where there could be bindings defined. It would be analogous to our ADM + representations mechanism in DID Core. In the Resolution spec, it would be the interfaces + bindings. Perhaps we do this if it becomes clear that volunteers for writing the tests and two independent implementations are not forthcoming by the end of the first Candidate Recommendation phase? I'd like to raise this concern on Tuesdays call.

[msporny]

@brentzundel wrote:

I believe this PR includes changes from PR #678. Not sure if that was deliberate.

Yes, that was deliberate... this PR is built on top of #678 because it needs those changes to make sense. Given that people have approved this PR, the assumption is that they're ok with #678 as well (but I will make doubly sure that's true by checking the reviews on both PRs. If there is agreement on #678, I'll merge that to main, and then retarget this PR to main and merge.

[msporny]

Substantive, multiple reviews, changes requested and made, no blocking objections (but a standing request from @OR13 exists to possibly define abstract functions), merging.