search

goss-org / goss

Quick and Easy server testing/validation
https://goss.rocks
Apache License 2.0
5.53k stars 470 forks source link

goss should not error if there are no tests defined #368

Closed gtmtech closed 4 years ago

gtmtech commented 5 years ago

Having an empty test file is a perfectly reasonable scenario (especially when another tool is generating said tests). Having goss error 1 for an empty tests file is unreasonable given that the test file has effectively passed and there are no failed tests. (empty, not missing - for missing its reasonable).

shalomb commented 5 years ago

I somewhat disagree with this request - only because we do a similar thing to have ansible generate the tests files and to have ansible automatically run goss to do validation right after. In our case, we always expect the tests file to be populated with tests and so for goss to fail our pipelines if no tests were run. This has actually saved our skins many a time in production. If this behaviour were inverted, we would need to have some other mechanism to check if no tests were populated to then fail the pipeline.

Perhaps a GOSS_NO_TESTS_MEANS_PASS variable to invert the behaviour would serve this request and not break existing users?

aelsabbahy commented 4 years ago

I'm going to close this for the reasons @shalomb mentioned. At this point I feel a few users depend on the current implementation as a safety check when using dynamically generated tests.

If you (or anyone else reading this) feel strongly about this being a feature, feel free to open a new feature request and I'll review adding an optional flag to allow empty test files to silently skip.