search

CleanCut / green

Green is a clean, colorful, fast python test runner.
MIT License
791 stars 75 forks source link

Enable minimum coverage option #135

Closed LeoHuckvale closed 5 years ago

LeoHuckvale commented 7 years ago

It would be useful with the coverage integration to be able to specify a minimum coverage level, under which the tests will fail.

As much as chasing coverage is a fools errand, for some projects it's necessary to set a high bar...

CleanCut commented 7 years ago

Does the coverage project have a built-in way to do such a thing?

Coverage is measured after all tests have run, so there is no particular test to tie the coverage to. How would you envision such a feature working?

LeoHuckvale commented 7 years ago

Sorry - I've just realised this is potentially a dupe of #131.

What I meant was, it would be good if Green could report the whole suite as failing (not a particular test), i.e. green should exit with non-zero status if coverage does not exceed the minimum coverage set in .coveragerc.

I'll look into the code if I get the chance and see if I can make a sensible pull req.

CleanCut commented 7 years ago

That could work. I look forward to seeing your PR.

lsh-0 commented 7 years ago

I would love for that feature to be a first class citizen.

I'm currently grepping the output in a bash script and exiting with a 1 if it's below a certain threshold. Feels completely hacky, but it gets the job done. If a min coverage param is supported in the rc file and was being picked up and passed to coverage that would be an elegant solution indeed.

I wish I had more time to delve into green and contribute PRs but test runners (from the history in your readme @CleanCut) sound like an arcane artform to understand and write.

I do have other features I would love to be present in green, like a global timeout for unittests (ie "if a test takes longer than x seconds, its a fail) and emitting metrics that I can send to our reporting db, but I don't have the foggiest idea where to begin. I made an attempt in the linked ticket but if a developer guide existed with some initial hand-holding ... the barrier to participating would be lowered.

CleanCut commented 5 years ago

I'd like to work on this feature the next time I get a little bit of spare time. I'd also be perfectly happy if someone beat me to it.

CleanCut commented 5 years ago

Included in 2.16.0 (just released) as -m/--minimum-coverage