search

juju-solutions / matrix

Automatic testing of big software deployments under various failure conditions
Other
8 stars 9 forks source link

Request: better define success test or failure #41

Closed pengale closed 7 years ago

pengale commented 7 years ago

I ran into this while writing a Zookeeper test.

Here's what I want to have happen:

1) We deploy a bundle, and wait until we are in a "healthy" state. 2) We run glitch. 3) We wait until we are in an "unhealthy" or "healthy" state. 4) We fail if unhealthy, and pass if healthy

Right now, there isn't a great way to define this in a matrix.yaml -- after the health check gets set to healthy, and the health task gets marked as "completed", it's hard to get it to check for health again.

More generally, as @bcsaller brought up, we need a formal way of defining a task success or failure, preferably with a way of waiting until the consequences of the test have settled out.

bcsaller commented 7 years ago

PR #51 should address at least part of this