search

App-vNext / Polly

Polly is a .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner. From version 6.0.1, Polly targets .NET Standard 1.1 and 2.0+.
https://www.thepollyproject.org
BSD 3-Clause "New" or "Revised" License
13.43k stars 1.23k forks source link

[Feature request]: Speed-up the build by executing the mutation tests in a separate workflow #1755

Closed martintmk closed 1 month ago

martintmk commented 1 year ago

Is your feature request related to a specific problem? Or an existing feature?

I think we can speed-up the build by extracting the mutation steps from https://github.com/App-vNext/Polly/blob/main/.github/workflows/build.yml into separate workflows.

Describe the solution you'd like

I believe we can introduce two new workflows:

This way the mutation tests is decreased by half on top of of reduced build times. With this change our PR builds could be done in less than 20 minutes.

Additional context

No response

martincostello commented 1 year ago

Suggest core not v8, then we don't need to rename it if we get to Polly 9.

rorypierce commented 1 year ago

So I should probably familiarize myself with the code base first... But as somebody trying to get a 10k meter perspective: would you be able to clarify a mutation test in the context of this particular project/build pipeline? I've done a reasonable amount of TDD and or PDD (pain driven development lol ) but I haven't seen the word "mutation" used in software parlance since I was taking genetic algorithms in undergrad (just to clarify my context). Thanks.

martincostello commented 1 year ago

The mutation tests are currently run by this cake task, which uses Stryker to run mutations on our different unit test projects:

https://github.com/App-vNext/Polly/blob/98f29c84d476f37d94b50ceac618494509a807e7/build.cake#L170

martintmk commented 1 year ago

but I haven't seen the word "mutation" used in software parlance since I was taking genetic algorithms in undergrad (just to clarify my context). Thanks.

Essentially, mutation tests are tests for unit tests :D

Copied from official docs:

TL; DR: Mutation testing introduces changes to your code, then runs your unit tests against the changed code. It is expected that your unit tests will now fail. If they don't fail, it might indicate your tests do not sufficiently cover the code.

https://stryker-mutator.io/docs/

ShawnWu33 commented 1 month ago

Hi @martintmk I am happy to take this if nobody is working on, Just want a bit clarification about the suggested solution as below. As I can see currently we ran following mutation tests: https://github.com/App-vNext/Polly/blob/4e7d099c30447c71e992d61aacf40a7616562bf6/build.cake#L179-L183 Is that only the last line Polly.proj belongs to legacy workflow and all the rest belongs to core workflow?