sixty-north / cosmic-ray

Mutation testing for Python
MIT License
556 stars 54 forks source link

Baseline creation throws an error when I skip a test in my test command #549

Closed cedaniel200 closed 1 month ago

cedaniel200 commented 1 month ago

I have the following configuration file:

[cosmic-ray]
module-path = ["application", "domain", "entrypoints", "infraestructure", "commons.py"]
timeout = 10.0
excluded-modules = []
test-command = "python3 -m pytest -k 'not integration' ../../tests/src/calculator"

[cosmic-ray.distributor]
name = "local"

This command allows me to skip some integration tests, said test runs correctly on my machine Command: python3 -m pytest -k 'not integration' ../../tests/src/calculator

When I try to run the baseline command cosmic-ray --verbosity=INFO baseline mt-calculator.toml`

If I remove the -k 'not integration' baseline command everything works correctly, but the contract/integration tests would be running and this is behavior that I do not want.

Any ideas on how to solve it, thanks in advance

abingham commented 1 month ago

I'm able to replicate this. It looks like the problem is in cosmic_ray.testing.run_tests(), line 45, where we call split() on the test command to pass it into subprocess.run(). This is too naive, resulting in "'not" and "integration'" being passed as individual arguments to the subprocess; we should instead be passing "'not integration'" as a single argument. I'll look into this.

abingham commented 1 month ago

Looks like we need to use shlex.split.

abingham commented 1 month ago

This should be fixed in 8.3.15, but let me know if it's still a problem. Thanks for reporting this!