Clarify Presentation Request Interior Requested_Attributes syntax and semantics

This is is related to https://github.com/hyperledger/aries-cloudagent-python/issues/155 In that it is the result of trying to create a proof of equality to a string using the "requested_attributes" field in the presentation request. Apparently the "requested_predicates" field in the presentation request only applies to integer valued fields in the credential and does not have an equality predicate, only <,>,<=,>= or range using two predicates?.

The documentation on how to use the requested_attributes and requested_predicates fields his sparse.

The best I could find is here but is frustratingly incomplete. https://github.com/hyperledger/aries-rfcs/tree/master/features/0037-present-proof

Through trial and error I was able to get several variants of a proof of name to work. It would be extremely helpful to get confirmation of what actually is being proven. In general explicit documentation of the syntax and semantics with seminal examples of proof requests would be most helpful.

The example is simple. Request proof that the name field in credential has a specific value. In this case "Alice Jones".

The two most relevant examples follow.

In this first example in the requested_attributes block a list of restrictions is provided. The document https://github.com/hyperledger/aries-rfcs/tree/master/features/0037-present-proof does not show restrictions as a field in the requested_attributes block but I found it in an example somewhere and it worked when the only restriction was the credentials_def_id. So I tried adding a value restriction. This worked and is explicit enough that I believe it is proving the the name is a string with the value "Alice Jones"

{
  "connection_id": "d965bd54-affb-4e11-ba8e-ea54a3214123",
  "version": "1.0.0",
  "name": "Proof of Name",
  "requested_attributes":
  [
    {
      "name": "name",
      "restrictions":
      [
        {
          "cred_def_id" : "3avoBCqDMFHFaKUHug9s8W:3:CL:13:default"
        },
        {
          "value": "Alice Jones"
        }
      ]
    }
  ],
  "requested_predicates":
  [

  ]
}

The document https://github.com/hyperledger/aries-rfcs/tree/master/features/0037-present-proof gives an example where is just includes multiple fields in the requested_attributes block including name, value, and credentials_def_id so I tried that. It worked but I am not sure what was proven. Is this equivalent to the example above. It is not clear.

{
  "connection_id": "d965bd54-affb-4e11-ba8e-ea54a3214123",
  "version": "1.0.0",
  "name": "Proof of Name",
  "requested_attributes":
  [
    {
      "name": "name",
      "value": "Alice Jones",
      "cred_def_id" : "3avoBCqDMFHFaKUHug9s8W:3:CL:13:default"
    }
  ],
  "requested_predicates":
  [

  ]
}

I also tried a mixture that also worked but not sure what was proven.

{
  "connection_id": "d965bd54-affb-4e11-ba8e-ea54a3214123",
  "version": "1.0.0",
  "name": "Proof of Name",
  "requested_attributes":
  [
    {
      "name": "name",
      "value": "Alice Jones",
      "restrictions":
      [
        {
          "cred_def_id" : "3avoBCqDMFHFaKUHug9s8W:3:CL:13:default"
        }
      ]
    }
  ],
  "requested_predicates":
  [

  ]
}

This fourth one does not restrict the value so I don't think it is proving that name equals "Alice Jones" but merely that the name field exists but maybe not.

{
  "connection_id": "d965bd54-affb-4e11-ba8e-ea54a3214123",
  "version": "1.0.0",
  "name": "Proof of Name",
  "requested_attributes":
  [
    {
      "name": "name",
      "restrictions":
      [
        {
          "cred_def_id" : "3avoBCqDMFHFaKUHug9s8W:3:CL:13:default"
        }
      ]
    }
  ],
  "requested_predicates":
  [

  ]
}

I looked at the presentation exchange records and found some fields that might explain what was proven but they are not documented so its merely a guess.

I think the revealed attributes shows what is proven but not sure

"requested_proof": {
      "revealed_attrs": {
        "a88aff02-454b-4256-b3de-09edde2620fb": {
          "sub_proof_index": 0,
          "raw": "Alice Jones",
          "encoded": "72896232743708443677449555551687504476536417389324439453514323796296385992918"
        }
      },
      "self_attested_attrs": {},
      "unrevealed_attrs": {},
      "predicates": {}
    },

"requested_proof": {
      "revealed_attrs": {
        "55cce48e-c037-4e11-8f97-8988056d57c1": {
          "sub_proof_index": 0,
          "raw": "Alice Jones",
          "encoded": "72896232743708443677449555551687504476536417389324439453514323796296385992918"
        }
      },
      "self_attested_attrs": {},
      "unrevealed_attrs": {},
      "predicates": {}
    },


 "requested_proof": {
      "revealed_attrs": {
        "c4e9d3e5-0143-4e1d-b7d6-ba49fb4b29a4": {
          "sub_proof_index": 0,
          "raw": "Alice Jones",
          "encoded": "72896232743708443677449555551687504476536417389324439453514323796296385992918"
        }
      },
      "self_attested_attrs": {},
      "unrevealed_attrs": {},
      "predicates": {}
    },

"requested_proof": {
          "revealed_attrs": {
            "f5d03520-f97b-4a95-9dca-562b76023877": {
              "sub_proof_index": 0,
              "raw": "Alice Jones",
              "encoded": "72896232743708443677449555551687504476536417389324439453514323796296385992918"
            }
          },
          "self_attested_attrs": {},
          "unrevealed_attrs": {},
          "predicates": {}
        },

Because all four are the same it makes me wary that I am not interpreting it correctly. I do not trust that I understand what is being proven. As it seems that if they are proving the same thing then there are 4 different ways to request the proof.

The full presentation exchange records for each of the four proof requests are given here

https://github.com/SmithSamuelM/leopy/blob/master/src/demo/FaberAliceCredential.md

Awesome stuff @SmithSamuelM. Especially the full sequence of data.

The immediate answer I have for you is not helpful: Questions and requests for changes about the behaviour of proof requests and proofs belong in the indy-sdk repo, not in aries. The reference material in aries-rfc 0037 is likewise intentionally vague and points to indy-sdk as a reference because in theory an Aries agent should/will be able to use the same proof request/proof messages for a different verifiable credentials model. That does not help the person trying to learn Aries and ACA-Py, and so I'll try to see what we can do to address that, without too much duplication of indy-sdk documentation/details.

The "restrictions" in indy-sdk proof requests are query parameters for finding credentials in the wallet that can be used for proofing the specific attribute (claim). Thus, since all of your proofs are based on restrictions that successfully return the single credential in your wallet all of them work. The restrictions are passed to a "WQL" (Wallet Query Language) routine in the indy-sdk that returns a set of references to credentials in the wallet that is then used in the indy-sdk code that generates the proof. Ideally, the controller ensures that if multiple credentials are returned, one is selected to be used for each attribute. For example a controller might show a user the list of credentials that could be used to satisfy the claim from the proof and allow the user to select one. Or, the controller might randomly select one from the set. Depends on the business requirements.

A predicate is processed after the a credential has been selected to satisfy the requested attribute (claim). The predicate is an expression that is evaluated to true or false using the encoded value.

As discussed last week, the "equal" predicate is (claimed to be) of marginal value in a ZKP since if you know the value already, proving it as an expression is not useful. The indy team did add "equals" as a "restriction", hence why it is now working when used there.

The encoding scheme is "controversial" (shall we say) in the current Indy. For predicates to work the issuer and the verifier must agree on the encoding, there is nothing in indy schema or in the indy-sdk on what that encoding should be. Some work has been done on trying to codify that, but it's not been fully fleshed out, last I checked. The Rich Schema (aka AnonCreds 2.0) work includes support for defining the encoding as part of what goes on the ledger, and that should address that issue.

Raw values accompany the encoded values, but are not used in the proof generation process. Done properly, the encoded value should derive from the raw value (so the verifier can check), and be able to be used as a predicate. At times achieving both of those requirements might be challenging. It is optional to include or not (aka reveal) the raw values. I think that is also a problem - the verifier should be be able to say what values must be revealed.

Self-attested attributes are not signed in the current implementation, which is not ideal. Evidently that is to be addressed in the AnonCreds 2.0 release.

Hope this helps.

@swcurran Much appreciate the thoughtful response. Your answer on the details was very informative but I am not sure if it answers my question =( I am still trying to understand the implications of your answer. I will dig deeper into the Indy SDK documentation, hopefully it will start to make sense. Based on the recent Hyperledger Indy Contributor and Aries working group calls, however, it seems that going forward, presentation proofs of Indy AnonCredsZKPs will be an Aires plugin not merely an Indy SDK library element. Intuitively, from a documentation perspective, an Aries cloud agent that exposes an API for presenting proofs should be discoverable and understandable from the ACA side. This means some significant distillation of the use of the proofs via the ACA-py interface. Especially because there is some modulation of the underlying libraries by the API. (ie some of the fields names and other things are slightly changed. Otherwise it becomes really difficult for new adopters of the Aries platform to get up to speed. I am working with a bunch of new devs to get up to speed on ACA but have frustration that the only way to actually understand how to use the meat of ACA (VC and presentation proofs for Indy AnonCreds) is to acquire what constitutes extensive background knowledge on the now deprecated Indy SDK. I get that asking for the ACA devs to provide a distillation of this information this is a heavy lift (especially for BC Gov) who are carrying most of the water. So help is always greatly appreciated But it appears that the the VC anoncreds from Indy SDK stuff at least from an Agent perspective will now be part of Aires. If Aries-rfc 0037 (and 0036) are intentionally vague then new RFCs that are not intentionally vague are needed to document in detail how to use indy anoncredes presentation proofs in ACA. One reason I am documenting in detail my explorations of the ACA-PY interface is help other devs get up to speed without repeating the trial and error.

Given that restrictions handle the equality contraints (was not obvious) then the predicates which are really limited to integer threshold predicates (in this sense the naming is confusing as restrictions are are type of predicate in the general definition of predicate) is understandable. I would have picked something like "thresholds" not predicates as a special case of restrictions.This naming confusion is a result of sparse documentation and trying to understand merely based on the field names. Now that I have some clarity, I suppose there is little need for an equality integer threshold predicate as one could create the equivalent of an equality integer threshold predicate by creating a range predicate with only one integer in the range or two threshold predicates where the combined predicates are both only true for one integer. Integer threshold predicates are inherently more understandable (at least on the surface) because they are much more limited in syntax and semantics. So understanding restrictions given the multiple ways to express them is the problem I am trying to solve..

I definitely agree on the need for documentation that gets the developer up to speed quickly is crucial, and you are right, it should be driven from Aries. We don't have the capacity to do that on our own - the team or absolute knowledge of the workings of indy. That has been an ongoing challenge in the community. We're getting a lot better.

It appears that the restrictions "value equals" implementation that was done recently in Indy has a bug in it - at least that's the current theory, as we are also struggling with what you are seeing. Nick was investigating that last night. We should know more today and will get the information to the Indy team if that's the case.

openwallet-foundation / acapy

Clarify Presentation Request Interior Requested_Attributes syntax and semantics #175