hyperledger-archives / aries-protocol-test-suite

Test Suite for testing protocol compliance of Aries Agents
https://wiki.hyperledger.org/display/aries
Apache License 2.0
25 stars 19 forks source link

Test selection and listing #14

Closed dbluhm closed 5 years ago

dbluhm commented 5 years ago

Select tests by test name (e.g. connections,1.0,inviter,can-start) and add listing option to command line arguments. Also includes some other minor tweaks such as keeping output brief when not capturing output, preventing interop report when using --collect-only, etc.

Sample test listing:

$ protocoltest --collect-only --list

Attempting to load configuration from file: .../aries-protocol-test-suite/config.toml

==== test session starts =====
platform linux -- Python 3.7.4, pytest-4.6.3, py-1.8.0, pluggy-0.12.0
rootdir: .../aries-protocol-test-suite, inifile: pytest.ini
plugins: metadata-1.8.0, json-report-1.1.0, asyncio-0.10.0
collected 3 items
<Package .../aries-protocol-test-suite/protocol_tests>
  <Module test_simple_messaging.py>
    <Function test_simple_messaging>
<Package .../aries-protocol-test-suite/protocol_tests/connection>
  <Module test_manual.py>
    <Function test_connection_started_by_tested_agent>
    <Function test_connection_started_by_suite>

---- Available Tests ----
[
  {
    "name": "simple,0.1,*,simple",
    "description": "Show simple messages being passed to and from the test subject."
  },
  {
    "name": "connections,1.0,inviter,can-start",
    "description": "Test a connection as started by the agent under test."
  },
  {
    "name": "connections,1.0,invitee,can-receive",
    "description": "Test a connection as started by the suite."
  }
]

==== no tests ran in 0.04 seconds ====