search

stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.78k stars 188 forks source link

Stryker failure : System.InvalidOperationException: Sequence contains no elements #713

Closed andrey-noskov closed 4 years ago

andrey-noskov commented 5 years ago

Describe the bug I don't have a clear enough description, but in Azure DevOps Stryker constantly fails on the "analyzing project" step and exits with code 1. May be I am missing something obvious.

Logs The tool is executed in Azure DevOps, so the log is copied from the step logs. stryker-log.log

Expected behavior Analyze the project and run mutation tests.

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

richardwerkman commented 5 years ago

Thanks for submitting this issue! Does this behaviour only occur on Azure Devops? It looks like a dependency of ours (buildalyzer) cannot open your project file CoreTests\CoreTests.csproj.

I'm curious if this happens too if you run stryker locally.

rouke-broersma commented 5 years ago

It looks like we are failing to read your test project file for some reason. Does this only happen on azure devops? Could you try it locally and use -f to produce the logfile?

Could you also perhaps share the project file with us?

andrey-noskov commented 5 years ago

yes, it fails in Azure Devops only. it works fine on my machine (the log is attached), though we seem to have a flaky test log-20190911.txt

andrey-noskov commented 5 years ago

and here is the project file with tests CoreTests.csproj.txt

richardwerkman commented 5 years ago

One difference I can think of is the msbuild version. There might be a version installed on DevOps that is not compatible with buildalyzer.

Unfortunately we don't log the msbuild version before stryker crashes... So I can't compare the versions.

andrey-noskov commented 5 years ago

is there any script or a command I can execute locally and on the agent to collect additional details?

rouke-broersma commented 5 years ago

If you add -f to the stryker run in your pipeline you could use publish pipeline artifact to publish the stryker logs to your build pipeline. You can then download them and send them to us.

Other than that I'm not sure yet. We are failing here:

https://github.com/stryker-mutator/stryker-net/blob/ccebdeecb0020f0a9010db2960c789a45cc783db/src/Stryker.Core/Stryker.Core/Initialisation/ProjectFileReader.cs#L58

Which to me indicates that for some reason we are unable to build your targetframework(s) using buildalyzer.

I'll try to repro using your csproj. Could you perhaps share the solutionfile and project under test projectfile as well? You can remove any sensitive information and I don't think I need any source files.

Also, which vm images are you using in your hosted pipelines? Just windows-latest or some other version? And are they the normal versions everyone else uses or are they some microsoft internal agents?

andrey-noskov commented 5 years ago

Here is the log from the agent: log-20190913.txt

the solution file: AsyncMediaShared.sln.txt

the project I want to test: Core.csproj.txt

The agents are internal, most likely with the latest Windows and pre-installed VS2017 and Azure SDK

rouke-broersma commented 5 years ago

I'm not seeing anything at all suspicious about those project files, SDK style and using package reference is pretty much ideal for buildalyzer and stryker (short of being dotnet core). Have you tried it with any other projects and/or agents?

andrey-noskov commented 5 years ago

yeah, I did. it fails with the same exception:

[07:15:03 ERR] An error occurred during the mutation test run System.InvalidOperationException: Sequence contains no elements at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)

rouke-broersma commented 5 years ago

Would it be possible to get access to the build agent or perhaps a list of all software and versions so I can try to reproduce? Software versions for windows, visual studio, msbuild, nuget and dotnet (core) installed on the system as a minimum to try to rebuild the agent on a vm.

rouke-broersma commented 4 years ago

Closing until repro is available

samvandenbossche commented 4 years ago

Is there any progress on this issue? I'm having the exact same behaviour but on my local machine

richardwerkman commented 4 years ago

@samvandenbossche Do you have a reproduction project that we can use to analyse the problem? With the current info we cannot research what went wrong.

samvandenbossche commented 4 years ago

I'm afraid not because it's against employer guidelines. it's a mulpti-project solution ( also multiple test projects ) All projects have target framework 4.7.2 and are using the new Dependency format. MsBuild version 4.8.3752.0

rouke-broersma commented 4 years ago

Could you perhaps create a reproduction using your current project as a template? Just remove anything that's not necessary to produce the failure, anything proprietary. I know that might be a lot of work but if you can't help us create a reproduction there's unfortunately nothing we can do to investigate.

samvandenbossche commented 4 years ago

I'll give it a try

eNeRGy164 commented 4 years ago

If the builds were running under Linux, this issue might have fixed it: https://github.com/daveaglick/Buildalyzer/pull/142 This is part of Buildalyzer 3.0.1

rouke-broersma commented 4 years ago

Thanks for the ping @eNeRGy164.

I don't think they were running on linux as they were all dotnet framework projects as far as I can tell but I'm gonna go and close this issue anyway since there hasn't been any activity for a while.