search

hyperledger-archives / aries-protocol-test-suite

Test Suite for testing protocol compliance of Aries Agents
https://wiki.hyperledger.org/display/aries
Apache License 2.0
25 stars 19 forks source link

eliminate ordering #8

Closed dhh1128 closed 5 years ago

dhh1128 commented 5 years ago

Generally, test suites avoid constructs that require ordering because they can lead to unreliable reports if the order is short-circuited. I think we need to remove all support for ordering (the priority feature) and replace it with reliable setup/fixtures.

dbluhm commented 5 years ago

I'm not really married to the idea and am happy to remove it but ordering features wasn't intended as a way of fulfilling dependencies. It made sense to me, for instance, that the core protocol tests would run before secondary or non-core protocols even though each test is completely independent from the others.

dbluhm commented 5 years ago

One of my secondary goals for the test suite was displaying the flow of messages, if you choose to, as a way to get familiar with what is expected behavior for a protocol. Ordering the protocols makes more sense with this goal in mind and less when all you want out of it is the report at the end.

dbluhm commented 5 years ago

14 takes care of this