search

boxed / mutmut

Mutation testing system
https://mutmut.readthedocs.io
BSD 3-Clause "New" or "Revised" License
902 stars 109 forks source link

--CI parameter #309

Closed enalcopolo closed 6 months ago

enalcopolo commented 6 months ago

Where can I find information about the --CI parameter? I want to integrate mutmut to a jenkins pipeline Thanks

boxed commented 6 months ago

I would argue against it. Mutation testing makes no sense in CI in my opinion. It's a tool you use to write tests.

enalcopolo commented 6 months ago

For me they help a lot in improving the quality of the code, therefore to have a product to deploy, they can give you a score (killed mutants/total mutants) that can serve as a metric if you compare it with a threshold in a CI. I come back and ask

Why is the --CI parameter in mutmut, what is it for?

Thanks

boxed commented 6 months ago

@maxkahan do you want to answer this? Have you actually used this?

As the original author of mutmut I think this flag is a bad idea, a waste of time and electricity. 🤷

enalcopolo commented 6 months ago

Don't worry, I already understood what CI is for, in my pipeline, when I run mutmut run Always returned status code 6 and the build failed, but after running mutmut run --CI works perfect I will manage to get a score with the results.

Thanks

maxkahan commented 6 months ago

Hi @boxed yes I have used this, see for example this GitHub Actions workflow. I don't use it as part of a pipeline, but to validate my tests are sensible when I release larger features (I'm aware that's not the same way you use it).

Also, I've given several conference talks introducing developers to mutation testing, and every time I get questions about running in a CI system from Python developers. Now I have a sensible answer using mutmut with the caveat that you don't want to run mutation tests as part of your release pipeline. There are definitely advantages to being able to run lengthy processes on other machines.

So I think it should stay! It's your project though, so do as you see fit 🙂

@enalcopolo yes that's exactly why I wrote it, glad it's helpful to you too! 🙂