search

openwallet-foundation / owl-agent-test-harness

Aries agent test framework, with agent backchannel support
https://aries-interop.info
Apache License 2.0
60 stars 66 forks source link

Update manage to stop on agent failure #886

Closed nodlesh closed 2 weeks ago

nodlesh commented 3 weeks ago

When running a set of tests, and the start routine fails to communicate to one of the designated agents, the execution of the tests would continue. Of course this would cause some or all of the tests to fail. This PR changes that logic and when the test harness does not get a successful startup of an agent, will let the user know there was an issue, skip the running of the tests, and stop any running test harness services. This should end the interop results getting complete failures in the reports when it was just an agent startup issue.

nodlesh commented 2 weeks ago

There is an issue with the credo agent I've been working on for a few days now. So this PR stopped the running of tests last night. However the job to push results to Allure ran and it shows empty in the report. Do you think this is ok, or should we skip the results push if tests don't run. It is an indicator that something went wrong and no tests were executed I suppose. I'm leaning to avoiding a results push if tests are skipped. Agreed?

swcurran commented 2 weeks ago

Yes, I think that makes sense.