Write mutation tests

[gritcsenko]

What

• small and isolated modifications to production code
  • each modified version is called a mutant
• run (impacted) unit tests against each mutant
  • at least one failing test will kill the mutant
  • no failing tests let mutant survive
• effectiveness of test suites is measured by percentage of killed mutants

A mutant survives if it is not covered by an assertion. Kill all the mutants.

Why

• support test-driven development
• code coverage VS test effectiveness

Code coverage

• fast
• test-driven development
  • live unit testing
  • NCrunch for Visual Studio
  • Continuous testing + cover new and outdated tests (Rider)

Mutation testing

• slow
• thorough

When

• preferably on CI pipeline with ability to run locally

How

• mutate source code
• mutate byte code
• mutant schemata / mutation switching

Tools

• Awesome Mutation testing
• NinjaTurtles
• VisualMutator
• Fettle
• Testura
• Faultify
• Stryker.NET / Stryker

Conclusion

The good

• complements TDD
• complements Code Coverage reports
• library code

The bad

• no stand-alone metric for code quality
• app code
• infrastructure as code

The ugly

• depends on available mutators
• depends on mutable statements in production code
• may be long-running

[gritcsenko]

Stryker.NET

Install

.NET Core tool via NuGet.org dotnet-stryker

.NET Core global tool

dotnet tool install -g dotnet-stryker

.NET Core local tool

dotnet new tool-manifest

dotnet tool install dotnet-stryker

dotnet tool restore

Use

dotnet stryker --config-file-path ./stryker-config.json

dotnet stryker --help

Configuration

Reporters

Integration

Global and local tools

• Install a global tool
• Install a local tool

[gritcsenko]

Video for reference

[gritcsenko]

Repo F0-Talks-MutationTesting to look at examples