Fix broken regression test execution

[nodlesh]

My Peer DID tests broke the scheduled regression tests. This PR fixes that, and also fixes a small problem in the Peer DID tests themselves.

[swcurran]

Hmmm...weird.

When I try to do a rebuild off of main and this PR (command ./manage runset acapy -r), I get this error. Seems unrelated to what you are doing here. Any ideas?

@WadeBarnes - FYI.

22.46 Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
22.52 Executing: /tmp/apt-key-gpghome.C3EjzId5Am/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88
23.36 gpg: keyserver receive failed: No data
------
Dockerfile.acapy-main:3
--------------------
   2 |     
   3 | >>> RUN apt-get update \
   4 | >>>    && apt-get install -y git gnupg2 software-properties-common curl \
   5 | >>>    && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88 \
   6 | >>>    && add-apt-repository 'deb https://repo.sovrin.org/sdk/deb bionic stable' \
   7 | >>>    && apt-get update \
   8 | >>>    && apt-get install -y libindy libnullpay
   9 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update    && apt-get install -y git gnupg2 software-properties-common curl    && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88    && add-apt-repository 'deb https://repo.sovrin.org/sdk/deb bionic stable'    && apt-get update    && apt-get install -y libindy libnullpay" did not complete successfully: exit code: 2
Docker image build failed.

[swcurran]

Seems to be still broken after loading this PR -- see result of same command for main below:

Ran ./manage runset acapy -r, and got this error:

    And "Acme" sends the request to "Bob"                                                            # features/steps/0023-did-exchange.py:258 0.056s
      Traceback (most recent call last):
        File "/usr/local/lib/python3.7/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/usr/local/lib/python3.7/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "features/steps/0023-did-exchange.py", line 272, in step_impl
          if context.peer_did_method and context.peer_did_method != "unqualified":
        File "/usr/local/lib/python3.7/site-packages/behave/runner.py", line 321, in __getattr__
          raise AttributeError(msg)
      AttributeError: 'Context' object has no attribute 'peer_did_method'

    And "Bob" receives the request                                                                   # None
    And "Bob" sends a response to "Acme"                                                             # None
    And "Acme" receives the response                                                                 # None
    And "Acme" sends complete to "Bob"                                                               # None
    Then "Acme" and "Bob" have a connection                                                          # None

Seems to be better than main, where I get a lot of these errors running the same command:

      Traceback (most recent call last):
        File "/usr/local/lib/python3.7/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/usr/local/lib/python3.7/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "features/steps/0160-connection.py", line 95, in step_impl
          if row["start_parameters"]:
        File "/usr/local/lib/python3.7/site-packages/behave/model.py", line 1600, in __getitem__
          raise KeyError('"%s" is not a row heading' % name)
      KeyError: '"start_parameters" is not a row heading'

[swcurran]

The apt-key GPG error above seems to have been intermittent. Working fine now.

[swcurran]

Seems to be getting better -- lots more passing on this one with the same command ./manage runset acapy -r. Still some issues, such as:

  In order establish a connection with two parties,
  As a responder or a requester,
  I want to use DID Exchange(RFC0023) and Out of Band(RFC0434) protocols to accomplish this.
  @T001-RFC0023 @critical @AcceptanceTest
  Scenario: Establish a connection with DID Exchange between two agents with an explicit invitation  # features/0023-did-exchange.feature:9
    Given we have "2" agents                                                                         # features/steps/0160-connection.py:22 0.000s
      | name | role      |
      | Acme | requester |
      | Bob  | responder |
    When "Bob" sends an explicit invitation to "Acme"                                                # features/steps/0023-did-exchange.py:40 0.124s
    And "Acme" receives the invitation                                                               # features/steps/0023-did-exchange.py:219 0.158s
    And "Acme" sends the request to "Bob"                                                            # features/steps/0023-did-exchange.py:258 0.048s
    And "Bob" receives the request                                                                   # features/steps/0023-did-exchange.py:310 0.000s
    And "Bob" sends a response to "Acme"                                                             # features/steps/0023-did-exchange.py:355 1.077s
      Traceback (most recent call last):
        File "/usr/local/lib/python3.7/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/usr/local/lib/python3.7/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "features/steps/0023-did-exchange.py", line 377, in step_impl
          if context.peer_did_method and context.peer_did_method != "unqualified":
        File "/usr/local/lib/python3.7/site-packages/behave/runner.py", line 321, in __getattr__
          raise AttributeError(msg)
      AttributeError: 'Context' object has no attribute 'peer_did_method'

[WadeBarnes]

Hmmm...weird.

When I try to do a rebuild off of main and this PR (command ./manage runset acapy -r), I get this error. Seems unrelated to what you are doing here. Any ideas?

@WadeBarnes - FYI.

22.46 Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
22.52 Executing: /tmp/apt-key-gpghome.C3EjzId5Am/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88
23.36 gpg: keyserver receive failed: No data
------
Dockerfile.acapy-main:3
--------------------
   2 |     
   3 | >>> RUN apt-get update \
   4 | >>>    && apt-get install -y git gnupg2 software-properties-common curl \
   5 | >>>    && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88 \
   6 | >>>    && add-apt-repository 'deb https://repo.sovrin.org/sdk/deb bionic stable' \
   7 | >>>    && apt-get update \
   8 | >>>    && apt-get install -y libindy libnullpay
   9 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update    && apt-get install -y git gnupg2 software-properties-common curl    && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88    && add-apt-repository 'deb https://repo.sovrin.org/sdk/deb bionic stable'    && apt-get update    && apt-get install -y libindy libnullpay" did not complete successfully: exit code: 2
Docker image build failed.

The was an issue with the Ubuntu Key servers yesterday for a few hours.

[swcurran]

Ran the command with the latest update and things seem a lot better, but after a bunch of tests passed (and the tests at the bottom mostly passed), there were a number of tests in the middle that failed as follows. @nodlesh — Are these a result of this and the previous PR, or are these “expected” and should be dealt with separately?

I’ve not seen these “server got into trouble” messages in previous ACA-Py to ACA-Py test runs.

@RFC0025 @UsesCustomParameters @AIP10 @AIP20
Feature: RFC 0025 DIDComm Transports # features/0025-didcomm-transports.feature:2
  In order to communicate with other agents,
  As an Agent
  I want to create connections using different transport protocols.
  @T001-RFC0025 @AcceptanceTest @DIDExchangeConnection @Transport_Http @critical
  Scenario Outline: Create DIDExchange connection between two agents with overlapping transports -- @1.1 DIDExchange connection with both agents using HTTP for inbound and outbound transport  # features/0025-didcomm-transports.feature:23
    Given we have "2" agents                                                                                                                                                                    # features/steps/0160-connection.py:22 0.000s
      | name | role      |
      | Acme | responder |
      | Bob  | requester |
    And "Acme" is running with parameters "{"inbound_transports": ["http"], "outbound_transports": ["http"] }"                                                                                  # features/steps/0044-mime-types.py:12 31.508s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["http"], "outbound_transports": ["http"] }"                                                                                   # None
    When "Acme" and "Bob" create a new connection                                                                                                                                               # None
    Then the invitation serviceEndpoint should use the "["http"]" protocol scheme                                                                                                               # None

  @T001-RFC0025 @AcceptanceTest @DIDExchangeConnection @Transport_Ws @Transport_Http @normal
  Scenario Outline: Create DIDExchange connection between two agents with overlapping transports -- @3.1 DIDExchange connection with both agents using WS for inbound and outbound transport, but also supporting http for outbound transport  # features/0025-didcomm-transports.feature:35
    Given we have "2" agents                                                                                                                                                                                                                   # features/steps/0160-connection.py:22 0.000s
      | name | role      |
      | Acme | responder |
      | Bob  | requester |
    And "Acme" is running with parameters "{"inbound_transports": ["ws"], "outbound_transports": ["ws", "http"] }"                                                                                                                             # features/steps/0044-mime-types.py:12 31.497s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["ws"], "outbound_transports": ["ws", "http"] }"                                                                                                                              # None
    When "Acme" and "Bob" create a new connection                                                                                                                                                                                              # None
    Then the invitation serviceEndpoint should use the "["ws"]" protocol scheme                                                                                                                                                                # None

  @T001-RFC0025 @AcceptanceTest @DIDExchangeConnection @Transport_Http @Transport_Ws @normal
  Scenario Outline: Create DIDExchange connection between two agents with overlapping transports -- @4.1 DIDExchange connection with both agents using HTTP and WS for inbound and outbound transport  # features/0025-didcomm-transports.feature:40
    Given we have "2" agents                                                                                                                                                                           # features/steps/0160-connection.py:22 0.000s
      | name | role      |
      | Acme | responder |
      | Bob  | requester |
    And "Acme" is running with parameters "{"inbound_transports": ["http", "ws"], "outbound_transports": ["http", "ws"] }"                                                                             # features/steps/0044-mime-types.py:12 31.484s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["http", "ws"], "outbound_transports": ["http", "ws"] }"                                                                              # None
    When "Acme" and "Bob" create a new connection                                                                                                                                                      # None
    Then the invitation serviceEndpoint should use the "["http", "ws"]" protocol scheme                                                                                                                # None

  @T001-RFC0025 @AcceptanceTest @DIDExchangeConnection @Transport_Http @Transport_Ws @normal
  Scenario Outline: Create DIDExchange connection between two agents with overlapping transports -- @4.2 DIDExchange connection with both agents using HTTP and WS for inbound and outbound transport  # features/0025-didcomm-transports.feature:41
    Given we have "2" agents                                                                                                                                                                           # features/steps/0160-connection.py:22 0.000s
      | name | role      |
      | Acme | responder |
      | Bob  | requester |
    And "Acme" is running with parameters "{"inbound_transports": ["ws", "http"], "outbound_transports": ["ws", "http"] }"                                                                             # features/steps/0044-mime-types.py:12 31.483s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["ws", "http"], "outbound_transports": ["ws", "http"] }"                                                                              # None
    When "Acme" and "Bob" create a new connection                                                                                                                                                      # None
    Then the invitation serviceEndpoint should use the "["ws", "http"]" protocol scheme                                                                                                                # None

  @T001-RFC0025 @AcceptanceTest @DIDExchangeConnection @Transport_Http @Transport_Ws @normal
  Scenario Outline: Create DIDExchange connection between two agents with overlapping transports -- @5.1 DIDExchange connection with one agent using http for inbound and the other using ws and both agents supporting ws and http outbound  # features/0025-didcomm-transports.feature:46
    Given we have "2" agents                                                                                                                                                                                                                  # features/steps/0160-connection.py:22 0.000s
      | name | role      |
      | Acme | responder |
      | Bob  | requester |
    And "Acme" is running with parameters "{"inbound_transports": ["http"], "outbound_transports": ["ws", "http"] }"                                                                                                                          # features/steps/0044-mime-types.py:12 31.479s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["ws"], "outbound_transports": ["ws", "http"] }"                                                                                                                             # None
    When "Acme" and "Bob" create a new connection                                                                                                                                                                                             # None
    Then the invitation serviceEndpoint should use the "["http"]" protocol scheme                                                                                                                                                             # None

  @T001-RFC0025 @AcceptanceTest @DIDExchangeConnection @Transport_Http @Transport_Ws @normal
  Scenario Outline: Create DIDExchange connection between two agents with overlapping transports -- @5.2 DIDExchange connection with one agent using http for inbound and the other using ws and both agents supporting ws and http outbound  # features/0025-didcomm-transports.feature:47
    Given we have "2" agents                                                                                                                                                                                                                  # features/steps/0160-connection.py:22 0.000s
      | name | role      |
      | Acme | responder |
      | Bob  | requester |
    And "Acme" is running with parameters "{"inbound_transports": ["ws"], "outbound_transports": ["ws", "http"] }"                                                                                                                            # features/steps/0044-mime-types.py:12 31.496s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["http"], "outbound_transports": ["ws", "http"] }"                                                                                                                           # None
    When "Acme" and "Bob" create a new connection                                                                                                                                                                                             # None
    Then the invitation serviceEndpoint should use the "["ws"]" protocol scheme                                                                                                                                                               # None

  @T002-RFC0025 @AcceptanceTest @RFC0160 @Transport_Http @critical
  Scenario Outline: Create 0160 connection between two agents with overlapping transports -- @1.1 0160 connection with both agents using HTTP for inbound and outbound transport  # features/0025-didcomm-transports.feature:65
    Given we have "2" agents                                                                                                                                                      # features/steps/0160-connection.py:22 0.000s
      | name | role    |
      | Acme | inviter |
      | Bob  | invitee |
    And "Acme" is running with parameters "{"inbound_transports": ["http"], "outbound_transports": ["http"] }"                                                                    # features/steps/0044-mime-types.py:12 31.484s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["http"], "outbound_transports": ["http"] }"                                                                     # None
    When "Acme" and "Bob" create a new connection                                                                                                                                 # None
    Then the invitation serviceEndpoint should use the "["http"]" protocol scheme                                                                                                 # None

  @T002-RFC0025 @AcceptanceTest @RFC0160 @Transport_Ws @Transport_Http @normal
  Scenario Outline: Create 0160 connection between two agents with overlapping transports -- @3.1 0160 connection with both agents using WS for inbound and outbound transport, but also supporting http for outbound transport  # features/0025-didcomm-transports.feature:77
    Given we have "2" agents                                                                                                                                                                                                     # features/steps/0160-connection.py:22 0.000s
      | name | role    |
      | Acme | inviter |
      | Bob  | invitee |
    And "Acme" is running with parameters "{"inbound_transports": ["ws"], "outbound_transports": ["ws", "http"] }"                                                                                                               # features/steps/0044-mime-types.py:12 31.510s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["ws"], "outbound_transports": ["ws", "http"] }"                                                                                                                # None
    When "Acme" and "Bob" create a new connection                                                                                                                                                                                # None
    Then the invitation serviceEndpoint should use the "["ws"]" protocol scheme                                                                                                                                                  # None

  @T002-RFC0025 @AcceptanceTest @RFC0160 @Transport_Http @Transport_Ws @normal
  Scenario Outline: Create 0160 connection between two agents with overlapping transports -- @4.1 0160 connection with both agents using HTTP and WS for inbound and outbound transport  # features/0025-didcomm-transports.feature:82
    Given we have "2" agents                                                                                                                                                             # features/steps/0160-connection.py:22 0.001s
      | name | role    |
      | Acme | inviter |
      | Bob  | invitee |
    And "Acme" is running with parameters "{"inbound_transports": ["http", "ws"], "outbound_transports": ["http", "ws"] }"                                                               # features/steps/0044-mime-types.py:12 31.490s
      Assertion Failed: resp_status 500 is not 200; 500 Internal Server Error

      Server got itself in trouble

    And "Bob" is running with parameters "{"inbound_transports": ["http", "ws"], "outbound_transports": ["http", "ws"] }"                                                                # None
    When "Acme" and "Bob" create a new connection                                                                                                                                        # None
    Then the invitation serviceEndpoint should use the "["http", "ws"]" protocol scheme                                                                                                  # None

[nodlesh]

Yes, my run just finished as well and I get the same thing. Investigating...

[nodlesh]

@swcurran I'm thinking this is an issue with acapy-main. To prove it 'm trying to run the tests with acapy instead of acapy-main, however I'm getting an error ModuleNotFoundError: No module named 'aries_askar' starting the agent. Are you able to build and run with acapy? ./manage build -a acapy

[swcurran]

Same error when I try locally. I got it to work by adding the following to the requirements.txt.latest file:

aries-askar
indy-credx
indy-vdr

Not sure that is the right way to solve it, but clearly those are not being picked up automagically. Given that we have deprecated the Indy SDK and default to the shared libraries (Askar/CredX/IndyVDR), that seems odd that we have to do that. And I think (but am not certain) that I see in the logs we are getting the libindy (aka the Indy SDK), which we shouldn’t be.

I’m not sure of how Python resolves requirements when (in the case) the aries-cloudagent is the only local requirements, and everything else should be picked up from its requirements.

[swcurran]

Once I got the acapy image built, I ran the same tests and it looks like it is giving the same errors.

FYI - to find the same tests to run, I did a runset dry-run using the -n option (no run) — ./manage runset acapy -n. That spits out the run set ./manage run command, and the I changed acapy-main to acapy.

[swcurran]

Lots more errors in running with the acapy image vs. acapy-main. Not sure why, but many tests at the end of the run failed.

[nodlesh]

FYI, this is something I've done in the restart agent routine. Those tests are not starting their agents with the proper settings as it was before. Working on it.

[nodlesh]

I ran the failing tests in the debugger and they now work with the latest commit. I am running ./manage runset acapy -r now and I'm expecting it to work.

[nodlesh]

./manage runset acapy -r tests passed completely. This should be good to go.