search

SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.24k stars 754 forks source link

Could not load file or assembly TechTalk.SpecRun during dotnet test #1567

Closed lukepuplett closed 3 years ago

lukepuplett commented 5 years ago

SpecFlow Version:

Used Test Runner

Version number:

Project Format of the SpecFlow project

.feature.cs files are generated using

Visual Studio Version

Are the latest Visual Studio updates installed?

.NET Framework:

Test Execution Method:

TeamCity and our build engineer's workstation.

dotnet test [path to csproj]

<SpecFlow> Section in app.config or content of specflow.json

# specflow.json
{

}

Issue Description

Tests are against an ASP.NET Core 2.1 web application. The tests make use of the TestServer in-memory web server which sets up the real runtime DI container. While doing this, the CLR types in the runtime module (whatever that module/assembly is when run under test, I don't know) are enumerated using RuntimeModule.GetTypes(). During this call, the runtime attempts to load a type (unknown) from the TechTalk.SpecRun assembly but the assembly binary file cannot be found.

This works on my home machine and on my work machine which have Visual Studio 2019 15.9 and the SpecFlow extension installed.

It fails on the TeamCity build server and on our build engineer's local PC.

I am going to try using Azure DevOps and a Linux build server and see what happens.

Will paste more information in a second post below.

Steps to Reproduce

Run dotnet test.

lukepuplett commented 5 years ago

This is the text of a question on Stack Overflow. I tend to exhaust SO and my own trial/error before resorting to GitHub issues.

https://stackoverflow.com/questions/56110032/specflow-3-for-net-core-test-run-error-on-build-server

Our TeamCity server cannot run the SpecFlow 3 tests for a .NET Core project.

My home and work workstations are able to run these tests.

It seems that during the creation of the ASP.NET Core in-memory TestServer web server instance, all the types are being enumerated as part of the DI container setup.

Could not load file or assembly 'TechTalk.SpecRun...

System.Reflection.ReflectionTypeLoadException : Unable to load one or more of the requested types.
Could not load file or assembly 'TechTalk.SpecRun, Version=3.0.0.0, Culture=neutral, PublicKeyToken=d0fc5cc18b3b389b'. The system cannot find the file specified.
Could not load file or assembly 'TechTalk.SpecRun.Common, Version=3.0.0.0, Culture=neutral, PublicKeyToken=d0fc5cc18b3b389b'. The system cannot find the file specified.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.RuntimeAssembly.get_DefinedTypes()
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
...
// Abridged here
...
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.ConfigureServices(IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.Initialize()
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at Microsoft.AspNetCore.TestHost.TestServer..ctor(IWebHostBuilder builder, IFeatureCollection featureCollection)

Searching my workstation I can find the following matching assembly.

FullName
--------
C:\Users\luke.puplett\.nuget\packages\specrun.runner\3.0.337\tools\net45\TechTalk.SpecRun.dll
C:\Users\luke.puplett\.nuget\packages\specrun.runner\3.0.337\tools\netcoreapp2.0\TechTalk.SpecRun.dll
C:\Users\luke.puplett\.nuget\packages\specrun.runner\3.0.337\tools\netcoreapp2.1\TechTalk.SpecRun.dll
C:\Users\luke.puplett\.nuget\packages\specrun.runner\3.0.337\tools\netcoreapp2.2\TechTalk.SpecRun.dll

My test project references this stuff. Note, SpecRun.Runner is the package that has the missing assembly though it is in tools above. Hmm...

    <PackageReference Include="SpecFlow" Version="3.0.199" />
    <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.0.199" />
    <PackageReference Include="SpecFlow.xUnit" Version="3.0.199" />
    <PackageReference Include="SpecRun.Runner" Version="3.0.337" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.1">

I'll keep struggling on and will report back.

I removed the feature.cs files from Git tracking so I'm looking into the idea that these files are still on the build server, not cleaned out.

Additional query words: ci cicd pipeline

SabotageAndi commented 5 years ago

I am currently working on an issue with SpecFlow+Runner and ASP.NET Core. When this is fixed, I will notify you, that you can try the newer version. Perhaps it also fixes you issue.

lukepuplett commented 5 years ago

Thanks - I end this engagement with this client on Friday so I won't be able to test it I'm afraid. My home machine is working fine, so to test I'd need access to the client's build server where the issue is. (sorry Andi).

EM1L commented 5 years ago

I am currently working on an issue with SpecFlow+Runner and ASP.NET Core. When this is fixed, I will notify you, that you can try the newer version. Perhaps it also fixes you issue.

I worked along with Luke, I will be able to test this issue and I shall let you know... I shall wait for your update to try the new version once available 👍

FirasConsultant commented 5 years ago

Hi all, I have the same issue as described above by @lukepuplett When I try to build my CI pipeLine I got this error:

'executor://specrun/executorV3.0.346': Exception has been thrown by the target of an invocation. Could not load file or assembly 'TechTalk.SpecRun.Common, Version=3.0.0.0, Culture=neutral, PublicKeyToken=d0fc5cc18b3b389b'. The system cannot find the file specified.

Any idea ?!?

SabotageAndi commented 5 years ago

I am still working on the issue. It was more complicated as anticipated. I hope to finish it this week.

atang8dxc commented 5 years ago

Any news? I'm trying to run this in Azure DevOps and I'm getting the same exception.

SabotageAndi commented 5 years ago

We had public holidays in Austria and I was ill. But finally a new version is on NuGet. It's 3.0.352. It has fixed for loading .NET Core assemblies and more logging.

@FirasConsultant, @EM1L & @atang8dxc: please update to the latest version and report back

FirasConsultant commented 5 years ago

Hello, Tested yesterday after updating the version to 3.0.352, but I still having the same issue. Waiting for a new version that fix this issue. Thanks

SabotageAndi commented 5 years ago

Ok, I will have a look at it in this sprint.

SabotageAndi commented 5 years ago

@FirasConsultant I had a look at it now. I wasn't able to reproduce it. I have a test project that starts a kestrel server in a BeforeScenario hook. It is this code: https://github.com/SabotageAndi/CoreWiki/blob/SpecFlowPlusRunnerDebugging/CoreWiki.Specs/Support/Hooks.cs

So that I can fix this issue, I have to little information. Best is, you try to create a small project, where you can reproduce the problem.

thomasjacksonsantos commented 5 years ago

Hello, I have install specflow with .net core and not work my tests.

[26/06/2019 17:46:23 Error] Ocorreu uma exceção ao chamar o executor 'executor://specrun/executorV3.0.352': Exception has been thrown by the target of an invocation. Could not load file or assembly 'TechTalk.SpecRun.Common, Version=3.0.0.0, Culture=neutral, PublicKeyToken=d0fc5cc18b3b389b'. O sistema não pode encontrar o arquivo especificado. [26/06/2019 17:46:23 Informational] ========== Run test finished: 0 run (0:00:00,6276763) ==========

SabotageAndi commented 5 years ago

@thomasjacksonsantos Could you create a small project where you reproduce this issue? It would help us to solve the issue. Thanks!

thomasjacksonsantos commented 5 years ago

@SabotageAndi I create new project with NUnit .Net Core and clicked right in mode debug, this worked. I followed the tutorial in https://specflow.org/getting-started/ But other projet make in MSTest, should It be problem ?

ZeinSleiman commented 5 years ago

@SabotageAndi I am having the same issue. I am using NUnit .NET Core. When using .Net Framework, I didnt have any issues. Any idea how I can dig more into this and figure what the issue might be.

In the bin/Debug/netcoreapp2.2 I found SpecFlowPlusRunner->netcoreapp2.2 I found the TeckTalk.SpecRun.Common dll and moved it to the Debug folder. I dont see the error anymore. The problem the tests dont run because I get Message=No matching step definition found for one or more steps. All these steps used to work fine before.

I dont need want to move the dll manually and this is only happening for me now on MAC. On Windows the path seems to be resolved.

sonudavidson commented 5 years ago

Hi @SabotageAndi, Do you have any update on this issue?

SabotageAndi commented 5 years ago

@sonudavidson No update. I didn't get a project to reproduce the issue. Without that, I can't make much.

bardeik commented 4 years ago

SpecFlowTest.zip Hi, the attached example fails. Followed the step by step guide, but with Visual Studio 2019.

bardeik commented 4 years ago

I have also tried to upgrade to Latest stable 3.0.xxx, with no luck

SabotageAndi commented 4 years ago

@bardeik I have looked at the example, and I am not sure what it has to do with this issue? All projects are .NET Core 3.0 projects, which are not yet supported and will be supported with SpecFlow 3.1 and SpecFlow+Runner 3.1.

Do I missed something?

bardeik commented 4 years ago

Sorry, worked when downgrading to .NET core 2.2.

FBorges14 commented 4 years ago

@bardeik I have looked at the example, and I am not sure what it has to do with this issue? All projects are .NET Core 3.0 projects, which are not yet supported and will be supported with SpecFlow 3.1 and SpecFlow+Runner 3.1.

Do I missed something?

Is there any plan when these new versions are going to be released?

SabotageAndi commented 4 years ago

@FBorges14 They are already available as pre-release on NuGet.org. Stable- Release will be in the next weeks. We have one bug to fix before we can release.

dessap commented 4 years ago

I managed to fix this by updating Specrun to the latest version and copying the missing dlls from the SpecRun subfolder into the project root folder and set it to copy to output path.

PeteRoberts commented 4 years ago

Hi, I came across this thread, as I am having a similar issue. Added Specflow tests in a .Net Core 3.1 test project. The tests run on my machine but fail in the TeamCity pipeline with:-

"Could not load file or assembly 'TechTalk.SpecRun.Common, Version=3.2.0.0, Culture=neutral, PublicKeyToken=d0fc5cc18b3b389b'."

Was anybody able to resolve this when they encountered it? Many thanks.

These are my packages:-

<PackageReference Include="SpecFlow" Version="3.1.95" />
<PackageReference Include="SpecFlow.NUnit" Version="3.1.95" />
<PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.1.95" />
<PackageReference Include="SpecRun.NUnit" Version="3.2.22" />
kalyanchakravarthys commented 4 years ago

I got Specflow working with dotnet core netcoreapp3.1 working with few changes, 1) Set my package references as below

<ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.5.0"/>
    <PackageReference Include="xunit" Version="2.4.1"/>
    <PackageReference Include="coverlet.collector" Version="1.2.0"/>
    <PackageReference Include="SpecFlow" Version="3.1.97"/>
    <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.1.97"/>
    <PackageReference Include="SpecFlow.xUnit" Version="3.1.97"/>
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0"/>
    <PackageReference Include="FluentAssertions" Version="5.4.2"/>
    <PackageReference Include="BoDi" Version="1.4.1"/>
  </ItemGroup>

2) Set below environment variable export MSBUILDSINGLELOADCONTEXT=1

Note: I used git bash terminal to set this environment variable and execute tests in a windows 10 machine.

MilosRisticQA commented 4 years ago

Wow, thanks a lot! This solution works, just did it on Linux. I used bash shell for the command.

SabotageAndi commented 3 years ago

We made a lot of fixed in assembly loading with the latest version of the Runner. Please open a new issue if these issues happen again with the latest version.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.