hyperledger / aries-agent-test-harness

Aries agent test framework, with agent backchannel support
https://aries-interop.info
Apache License 2.0
60 stars 65 forks source link

Need tests that cover proof requests for multiple AnonCreds with/without revocation #605

Open swcurran opened 1 year ago

swcurran commented 1 year ago

We need a to add a set of tests that handle testing proof request involving:

The intent of the tests should be the following, but I'm not sure how many individual tests are needed to accomplish this. I'm ok with a long test that executes a number of scenarios -- e.g. providing the presentation before revocation and after in a single test.

   Scenario Outline: Present Proof where the presentation includes two revocable credentials
      Given "3" agents
         | name  | role     |
         | acme | issuer |
         | Faber | verifier |
         | Bob   | prover   |
      And "Acme" and "Bob" have an existing connection
      # Separate Tests Variation: Each of credential one and two are revocable, not revocable
      And "Acme" issues a credential to "Bob"
      And "Acme" issues a second credential to "Bob"
      And "Faber" and "Bob" have an existing connection
      # Revocation interval not included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the proof verified
      # Revocation interval included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the proof verified
      # Revoke one of the credentials -- if revocable
      And "Acme" revokes the first credential
      # Revocation interval not included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the not verified proof
      # Revocation interval included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the not verified proof
      # Revoke second of the credentials - if revocable
      And "Acme" revokes the second credential
      # Revocation interval not included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the not verified proof
      # Revocation interval included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the not verified proof
swcurran commented 1 year ago

These tests are intended to cover issues in ACA-Py (2036 and 1651, and BC-Wallet issue 762.

We also may need to have the revocation tests also cover whether the revocation interval in the proof request is applied at the top level (applying to all referents) or at the referent level. However, that can wait until we have the specific tests working.

It would be good to have this test working with the Mobile Backchannel so that we can also test individual wallets for this test case.

nodlesh commented 1 year ago

@swcurran In this middle of the long scenario you state this.

     # Revoke one of the credentials -- if revocable
      And "Acme" revokes the first credential
      # Revocation interval not included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the not verified proof

Shouldn't Bob have the proof verified in this case since no interval is included, therefore the verifier doesn't care if it was revoked or not?

swcurran commented 1 year ago

@swcurran In this middle of the long scenario you state this.

     # Revoke one of the credentials -- if revocable
      And "Acme" revokes the first credential
      # Revocation interval not included in Proof request
      When "Faber" sends a request for proof presentation to "Bob"
      And "Bob" makes the presentation of the proof
      And "Faber" acknowledges the proof
      Then "Bob" has the not verified proof

Shouldn't Bob have the proof verified in this case since no interval is included, therefore the verifier doesn't care if it was revoked or not?

Interesting question. For verification to be accepted when a revocation is not requested, the holder would have to not include a non-revocation proof in the response. Alternatively, the holder provides the non-revocation proof and the verifier ignores it -- either before or after running the verifier.

Lets have the test go with assuming it is valid to get a revoked credential when revocation data is not requested, and then we'll see what tests pass and fail.

swcurran commented 1 year ago

Further, we can decide if it does fail, how it can be corrected.

nodlesh commented 1 year ago

In the fibonacci scale, if 1 is a spelling correction, this is about an effort level 8. Most of the steps can be reused from other test scenarios, most of the work may be in constructing the red data, proofs, and making it work with the mobile backchannel.

The large scenario above should be broken down to properly identify and run particular cases individually. The effort of 8 includes all scenarios below that are tagged critical and normal.

@critical
Scenario Outline: Holder has 2 revokable credentials issued, both are not revoked, and receives a proof of non-revocation with interval for both credentials
@normal
Scenario Outline: Holder has 2 revokable credentials issued, both are revoked, and receives a proof of non-revocation with interval for both credentials
@critical
Scenario Outline: Holder has 2 revokable credentials issued, one is revoked, and receives a proof of non-revocation with interval for both credentials
@critical
Scenario Outline: Holder has 2 revokable credentials issued, one is revoked, and receives a proof of non-revocation with interval for the revoked credential
@low @NegativeTest
Scenario Outline: Holder has 2 revokable credentials issued, one is revoked, and receives a proof of non-revocation with interval for the non-revoked credential
@normal
Scenario Outline: Holder has 2 credentials issued, one revokable, is not revoked, and receives a proof of non-revocation with interval for both credentials
@normal
Scenario Outline: Holder has 2 credentials issued, one revokable, that is revoked, and receives a proof of non-revocation with interval for both credentials
@low @NegativeTest
Scenario Outline: Holder has 2 credentials issued, none revokable, and receives a proof of non-revocation with interval for both credentials

@normal
Scenario Outline: Holder has 2 revokable credentials issued, both are not revoked, and receives a proof for both and non-revocation with interval for one credential
@normal
Scenario Outline: Holder has 2 revokable credentials issued, both are revoked, and receives a proof for both and non-revocation with interval for one credential
@normal
Scenario Outline: Holder has 2 credentials issued, one revokable, is not revoked, and receives a proof for both and non-revocation with interval for revokable credential
@low @NegativeTest
Scenario Outline: Holder has 2 credentials issued, one revokable, is not revoked, and receives a proof for both and non-revocation with interval for non-revokable credential
@critical
Scenario Outline: Holder has 2 credentials issued, one revokable, that is revoked, and receives a proof for both and non-revocation with interval for revokable credential
@low @NegativeTest
Scenario Outline: Holder has 2 credentials issued, one revokable, that is revoked, and receives a proof for both and non-revocation with interval for non-revokable credential
@low @NegativeTest
Scenario Outline: Holder has 2 credentials issued, none revokable, and receives a proof of non-revocation with interval for both credentials

@critical
Scenario Outline: Holder has 2 revokable credentials issued, both are not revoked, and receives a proof with no revocation interval
@critical
Scenario Outline: Holder has 2 revokable credentials issued, both are revoked, and receives a proof with no revocation interval
@low
Scenario Outline: Holder has 2 credentials issued, one revokable, is not revoked, and receives a proof with no revocation interval
@normal
Scenario Outline: Holder has 2 credentials issued, one revokable, that is revoked, and receives a proof with no revocation interval
@low
Scenario Outline: Holder has 2 credentials issued, none revokable, and receives a proof with no revocation interval

I believe these scenarios capture everything in the large scenario above, plus more, including some negative tests. Welcome review of the priorities and the scenarios themselves.

nodlesh commented 1 year ago

Another question is how should we categorize these? Inside the existing revocation test feature file for RFC-0183 and HIPE-0011, or should we create a separate feature file and/or tagged with @PR-1651 and tests could be named T001-PR1651?

swcurran commented 1 year ago

At least some of the "negative tests" are not really negative tests -- such as requesting a revocation interval for non-revocable credentials. Keep in mind that a verifier does not necessarily know whether a credential held by the holder is revocable or not, and so whether they request a revocation interval or not is independent of whether the holder has a revocable credential or not. Consider for example. Many universities issue the same "Degree" credential to students, some that are revocable, others that are not. An employer would not know if an applicant had a revocable credential or and so would always request a revocation interval (or not) and happily accept unrevocable credentials. Note that after receiving the presentation, the verifier would know if the credential was revocable, as they would know what CredDef the issuer was using.

I would call the tests RFC-0441-REV and reference this section of RFC 0441.

As an aside from this issue -- we should also have some tests that cover RFC-0441-DATES that covers this part of the same RFC 0441.

swcurran commented 1 year ago

It will really take 20 tests to cover this? Presumably we will also need 2 credentials and 4 presentation requests. The four presentations would request would all request claims from both credentials, and would have:

nodlesh commented 1 year ago

Yes, agreed some of those Negative Tests don't sound like negative now that I read them again. Won't add those tags when the tests are written.

We can probably combine a few to make the list shorter. For example, could do the non revoked test, then revoke, and use the same presentation, all in one scenario. Will start with the list above and as they are written some will naturally merge where it makes sense.