citrusframework / yaks

YAKS is a platform to enable Cloud Native BDD testing on Kubernetes
Apache License 2.0
83 stars 27 forks source link

Optimize failure() condition for pre/post steps #433

Open christophd opened 1 year ago

christophd commented 1 year ago

Pre/post steps support condition failure() which runs the step only on a failed test.

When running a test group with multiple tests the failure() condition is evaluated on the whole test suite. This leads to the situation that the failure() condition evaluates to true for the whole test suite once a single test has failed. So all succeeding tests will run the steps using the failed condition regardless of the individual test result (success/failed).

We should change that behavior so the pre/post steps that are bound to failure state are run only for failed tests. Succeeding tests without failure should not run the steps in their pre/post script section because of this.