search

stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.75k stars 176 forks source link

Customize build step #2364

Closed vadim-96 closed 1 week ago

vadim-96 commented 1 year ago

For a big solution with dozens of projects included, the command dotnet build for old projects may build with errors. As an option, you can build the projects using msbuild.exe directly and this will not cause errors.

What I propose: Add an option to dotnet stryker to customize the build tool like dotnet stryker --build-tool "msbuild | dotnet build"

richardwerkman commented 1 year ago

We have been thinking about a --skip-build option. Because if the project was compiled before stryker runs, there is no need for Stryker to build again anyway (except maybe make sure the code is compilable).

However, this is no easy feat as buildalyzer removes build output when analyzing a project. So we'd have to somehow prevent buildalyzer from doing that. Or restore the build output by somehow backing it up before running buildalyzer.

Your option for a custom build step would also work. But I'd rather see the skip build step because that requires little extra scenarios we have to cover. And the skip build option boosts performance in some pipelines as the project is build anyway in the pipeline (in most cases).

vadim-96 commented 1 year ago

Your suggestion about adding --skip-build sounds great. It'll be good if it happens in the next releases

rouke-broersma commented 1 year ago

Your suggestion about adding --skip-build sounds great. It'll be good if it happens in the next releases

I doubt it, it's a hard or maybe even impossible problem to solve.

richardwerkman commented 1 year ago

It's been a long-living issue at Buildalyzer, but preferably it's fixed over there. If that doesn't work out we can look for workarounds in Stryker. But I'd rather not

dupdob commented 1 year ago

Sorry to intervene here, but I feel like we are missing the elephant in the room here: Stryker needs the tested project(s) to be buildable in a Buildalyzer compatible way, so that it can build the mutated one. And Buildalyzer even have problems supporting every existing standard workflows. Maybe we could open an issue to discuss how Stryker could works WITHOUT Buildalyzer to see what solutions we could identify. For example, if Stryker could generate 100% error free mutations, it could write down those files and trigger whatever build workflow the project needs. This is of course a big if, and it implies the ability to overwrite non mutated source files, but it may be the only forward

rouke-broersma commented 1 year ago

Buildalyzer is not necessary to use Roslyn to compile, so I don't see why we would need to move to overwriting source files on disk. We also don't need buildalyzer for the rollback mechanism.

There have been prs to investigate exchanging buildalyzer for plain msbuild workspaces, they were done by a contributor. I recently closed the prs because we didn't do anything with them and they were just sitting there. If anyone's interested in switching out from buildalyzer you can find those prs. However I am extremely hesitant because I don't see any good reason why it would work better than buildalyzer Buildalyzer in essence just calls msbuild.

There's a lot of blood sweat and tears of working with msbuild encapsulated in buildalyzer, I would not wish for us to repeat their mistakes and end up in the same situation anyhow.

dupdob commented 1 year ago

So, let me restate what I mean: Stryker needs to get project's details via Buildalyzer (or some equivalent) to be able to build the project in memory. Along those data, we have the list of source files, the compilations options and the dependencies, including analyzers and source generators. Sometimes, Buildalizer fails to identify those properly due to some bugs or limitations; but custom tasks (such as grpc generators) cannot be reliably identified and performed in memory. Hence, the only way (I can think of) to handle any project that cannot be built in memory is to build them from disk, as normally as possible.

The main question is the trend: is buildalizer (or an equivalent) efficient enough to cover most projects or is there a risk to have more and more incompatible projects.

dupdob commented 1 week ago

Coming back to the original issue: latest Stryker versions try to build the project using msbuild when dotnet build fails. So I assume this fix this issue. feel free to reopen this issue if you still have problems