Define a set of @SmokeTest scenarios

[tdiesler]

Currently, there is no quick way to verify a defined set of smoke tests. I could for example assign the @SmokeTest to the test scenarios that are needed to cover the standard Alice, Faber, Acme, Thrift example.

[nodlesh]

I could be wrong, but smoke tests could be aries framework dependant. What might be a smoke test for one may not be for another, since one framework supports revocation and the other does not, or only supports AIP10. I'd be hard pressed to assign a global smoke test(s) that could apply to all. What is the use case? Could we not just reference a set of tests by their other tags and that would represent your smoke tests?

[tdiesler]

I'd say that boils down to whether there is a notion of a minimum set of functionality that all must support i.e. a set of smoke tests that are expected to pass for all agents at all times.

Smoke tests as a function of a specific agent could be defined as such e.g. @SmokeTestAcaPy, @SmokeTestAfgo, etc. There might not be a lot of value in the latter though

[nodlesh]

This might be a good topic to discuss in the next Aries Test Harness Users Group meeting, to get some opinions and consensus.

[ianco]

You can use a very small number of tests to get a (fairly) complete coverage.

For example in the revocation tests (https://github.com/hyperledger/aries-agent-test-harness/blob/main/aries-test-harness/features/0183-revocation.feature) if you run @T001-HIPE0011 it will do a connection, create a schema and cred def, issue and revoke a credential, and ask for a proof.

There are similar tests in the aca-py integration test suite, so if you're specifically testing aca-py then you can use those as well. (The aca-py tests start up new agents for each test, so they are useful for testing different aca-py configurations.)

[tdiesler]

Closing for now (need a better view of what the test suite can do)