Solution config option does not get used when it in a child of the run directory

[JackSteel97]

Describe the bug Specifying a path to the solution file does not work as expected when running stryker from a directory outside the same directory as the solution. Below, the Broken section defines the bug scenario, the Working section defines a very similar Scenario that works as expected

Working

With the following file structure:

.
└── bar-service/
    └── app/
        ├── src/
        │   └── <project folders>
        ├── stryker-config.json
        └── FooSolution.sln

and the stryker-config.json file like this:

{
    "stryker-config":{
        "solution": "FooSolution.sln", 
        "mutation-level": "Complete",
        "reporters": ["html", "progress", "markdown"],
        "additional-timeout": 10000,
        "thresholds": {
            "high": 80,
            "low": 50,
            "break": 0
        },
        "coverage-analysis": "perTestInIsolation"
    }
}

Running the following commands:

> cd bar-service/app
> dotnet stryker --verbosity trace --log-to-file

everything works as expected. Stryker logs that it picked up the correct option for the "SolutionPath" setting: <absolute_path>\\bar-service\\app\\FooSolution.sln and logs:

[15:45:46 INF] Identifying projects to mutate in \bar-service\app\FooSolution.sln. This can take a while. before continuing to find and run mutation tests against all projects in the solution correctly.

Broken

However, if instead the config file and working directory are moved one step up things don't go as smoothly. With the following file structure:

.
└── bar-service/
    ├── stryker-config.json
    └── app/
        ├── src/
        │   └── <project folders>
        └── FooSolution.sln

and the stryker-config.json file like this:

{
    "stryker-config":{
        "solution": "app/FooSolution.sln", 
        "mutation-level": "Complete",
        "reporters": ["html", "progress", "markdown"],
        "additional-timeout": 10000,
        "thresholds": {
            "high": 80,
            "low": 50,
            "break": 0
        },
        "coverage-analysis": "perTestInIsolation"
    }
}

Running the following commands:

> cd bar-service
> dotnet stryker --verbosity trace --log-to-file

Logs that the "SolutionPath" option is just app/FooSolution.sln (no absolute path prepended) and then logs the folowing:

[15:55:53 INF] Identifying project to mutate. [15:55:53 VRB] Scanned the directory \bar-service for *.csproj files: found [] [15:55:53 INF] Time Elapsed 00:00:00.0984222 Stryker.NET failed to mutate your project. For more information see the logs below: No .csproj file found, please check your project directory at \bar-service

If I purposefully mess up the Solution path given in the config file to something that doesn't exist like app/FooService.sln then it correctly logs:

Given path does not exist: app/FooService.sln

suggesting that in the broken scenario it can correctly find the solution file but then doesn't use it at all to resolve the source projects

Expected behavior Stryker should be able to be run from a directory that is a parent of the directory containing the .NET solution if the solution config is set correctly

Desktop (please complete the following information):

• OS: Windows
• Type of project: Core
• Framework Version .NET 6
• Stryker Version 3.10.0

[rouke-broersma]

We don't support running stryker this way. These are the supported scenarios: https://stryker-mutator.io/docs/stryker-net/operating-modes/

[JackSteel97]

Why isn't this supported? Is there a technical reason stryker has to be initially called from the same directory as the solution?

[rouke-broersma]

Why isn't this supported? Is there a technical reason stryker has to be initially called from the same directory as the solution?

It's not supported because it's not in the list of supported modes. There's probably no technical reason why it couldn't be supported but no one's put in the effort yet and no one has as far as I know made the case for why Stryker should support this.

Stryker works a lot with filepaths and filepaths are not trivial, so any alternatives we support can cause a tonne more work and or bugs. We have limited bandwidth, we can't support every scenario.

If you make a case for running Stryker this way and we agree with the reasoning we might support it in the future, but right now we don't.