search

coreos / ignition

First boot installer and configuration tool
https://coreos.github.io/ignition/
Apache License 2.0
839 stars 247 forks source link

Add ability to specify expected failures in blackbox negative testing #748

Open arithx opened 5 years ago

arithx commented 5 years ago

Feature Request

Desired Feature

Add functionality to be able to declare specific expected failure cases for negative testing rather than assuming that any failure is expected.

The current system allows potential regressions to be masked if the test run still fails for other reasons.

Other Information

Originally brought up in https://github.com/coreos/ignition/pull/746#discussion_r260448340

ajeddeloh commented 5 years ago

Any thoughts on how to do this without log parsing?

arithx commented 5 years ago

Not off the top of my head.

ajeddeloh commented 5 years ago

Misc thought: we could do better than what we do now and specify which stage should fail