Closed richardwerkman closed 5 years ago
I played around with this a little bit already. At a minimum:
However, there's probably more -- I seem to get fewer mutations than I'd expect on the toy .NET Framework project I was trying this with.
@kevinlyles Did you try the toy project with Stryker.NET? If so, it is correct that it could mutate the project? That would be great news. What framework version was the project?
About the fewer mutations, that could be because of the lack of mutators at the moment?
We'll keep in mind that a new project file reader has to be created for the .NET framework.
Yep, I had to make the changes I made above, but it did seem to mutate some of the code.
I know there aren't many mutators yet, but it seemed like it was missing things that even the existing mutators should have caught. I may have missed something, though.
For an initial setup we could try running the DDL's using vstest.exe. We can use the Visual Studio Test task for TFS/VSTS as a reference on how to pass specific arguments.
I would suggest manually running the command on your machine first to see what kind of output you get from vstest.exe if all tests pass or if a test fails. The performance will probably be quite bad if we simply run all tests using vstest.exe (we won't be able to use incremental test runs because the DDL's don't change).
Simply checking for the exit code of the command would be fine to begin with. I'd prefer to have basic support over having no support at all. We may be able to determine the vstest.exe path based on the environment variables. The TFS/VSTS task is also able to do this, since it has a Visual Studio version 'Latest'.
I don't think we'll have any issues with mutating the code but it's good to also try that out in the early stage (we can simply disable/remove the initial test run to test this)
Today at the Stryker hackathon we've had a discussion about how to support .NET Framework. Our conclusion was that the two biggest changes we would have to make are:
An possible sollution for resolving the VSTest executable location can be found here https://github.com/cake-build/cake/issues/2077
We have looked at fettle for inspiration about how to execute Stryker as a .NET Framework executable. We now think installing using the package manager console is a good idea. After installing the executable can be executed through this console.
How about keeping the current CLI for .NeT Core and making a new one for .NET Framework? That fixes multiple issues at once.
@simondel Creating seperate CLI projects won't fix the issues we are having. And we would lose a lot of reusable code. Right now we are experimenting with a CLI that targets multiple runtimes.
Full Framework doesn't work with transient dependencies. This breaks resolving referenced assemblies. One workaround could be to move the msbuild task to the "project under test".
Another workaround could be to write a custom resolver for full framework. This resolver can read the .csproj
file and analyse the xml.
Useful articles for building NuGet Tool Installer compatible with both dotnet core and dotnet framework
http://www.marcusoft.net/2011/12/creating-tools-only-nuget-package.html https://natemcmaster.com/blog/2017/11/11/build-tools-in-nuget/
I can now find all the csproj files and load all its references. When I try to compile this code right now I get the following errormessage: Predefined type 'System.String' is not defined or imported
This means it did not load the mscorlib. I cant seem to find why it did not load, is there anyone who knows the solution to this? I found another issue on the roslyn github page: https://github.com/dotnet/roslyn/issues/12393
The code that tries to compile is as follows:
syntaxTrees: syntaxTrees,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary),
references: _input.AssemblyReferences);
// first try compiling
var emitResult = compiler.Emit(ms);```
Maybe it has to do with the way you resolved the AssemblyReferences? I don't know about your implementation but it could be that it always misses mscorlib
(and only mscorlib). In that case we could add it hardcoded on .net framework from the default location. I guess it's fair to assume that every .net framework project needs mscorlib
as a referenced assembly.
@yaelkeemink Is working on this on the _.NET_Framework_support
branch
I think I came across something similar re: the System.String
/ mscorlib
issue.
What worked for me in the end was using the Microsoft.Build.Locator nuget package (github repo).
E.g.
internal static class MSBuildWorkspaceFactory
{
static MSBuildWorkspaceFactory()
{
MSBuildLocator.RegisterDefaults();
}
public static MSBuildWorkspace Create()
{
return MSBuildWorkspace.Create(
new Dictionary<string, string>
{
{ "CheckForSystemRuntimeDependency", "true" }
});
}
}
and then
using (var workspace = MSBuildWorkspaceFactory.Create())
{
var solution = await workspace.OpenSolutionAsync(SolutionFilePath);
// change code then compile solution...
}
I've seen online that CheckForSystemRuntimeDependency
can also help so added it just in case ¯_(ツ)_/¯
@oliwennell Thanks for the suggestion! We'll see if we can use this to fix our issues locating dependencies in GAC.
What needs to be done to add support for .NET Framework next to .NET core support For now: