search

SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.22k stars 752 forks source link

SpecFlow 3.6.2 doesn't work with .NET Core 3.1.301 SDK and doesn't appear in ReSharper at all #2282

Open EtharAlali opened 3 years ago

EtharAlali commented 3 years ago

SpecFlow Version:

Used Test Runner

Version number: 3.17.0

Project Format of the SpecFlow project

.feature.cs files are generated using

Visual Studio Version

Enable SpecFlowSingleFileGenerator Custom Tool option in Visual Studio extension settings

Are the latest Visual Studio updates installed?

.NET Framework:

Test Execution Method:

<SpecFlow> Section in app.config or content of specflow.json

Issue Description

When creating a basic project and running Unit Tests, SpecFlow does not run and causes an error. Even using a basic calculator project. Unit tests work fine without SpecFlow and it can be made to work without ReSharper (ReSharper Test Runner is just a wrapper around NUnit, so in older versions of SpecFlow, 1.x.y to 2.x.y this just worked out of the box).

Errors:


: error MSB4018: The "GenerateFeatureFileCodeBehindTask" task failed unexpectedly.
: error MSB4018: System.Exception: Unit test Provider already set.
: error MSB4018:    at TechTalk.SpecFlow.UnitTestProvider.UnitTestProviderConfiguration.UseUnitTestProvider(String unitTestProviderName)
: error MSB4018:    at TechTalk.SpecFlow.NUnit.Generator.SpecFlowPlugin.GeneratorPlugin.Initialize(GeneratorPluginEvents generatorPluginEvents, GeneratorPluginParameters generatorPluginParameters, UnitTestProviderConfiguration unitTestProviderConfiguration) in D:\a\1\s\Plugins\TechTalk.SpecFlow.NUnit.Generator.SpecFlowPlugin\GeneratorPlugin.cs:line 14
: error MSB4018:    at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.LoadPlugins(ObjectContainer container, GeneratorPluginEvents generatorPluginEvents, UnitTestProviderConfiguration unitTestProviderConfiguration, IEnumerable`1 generatorPlugins)
: error MSB4018:    at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.CreateContainer(SpecFlowConfigurationHolder configurationHolder, ProjectSettings projectSettings, IEnumerable`1 generatorPluginInfos, IObjectContainer parentObjectContainer)
: error MSB4018:    at SpecFlow.Tools.MsBuild.Generation.WrappedGeneratorContainerBuilder.BuildGeneratorContainer(SpecFlowConfigurationHolder specFlowConfigurationHolder, ProjectSettings projectSettings, IReadOnlyCollection`1 generatorPluginInfos, IObjectContainer rootObjectContainer)
: error MSB4018:    at SpecFlow.Tools.MsBuild.Generation.GenerateFeatureFileCodeBehindTaskExecutor.Execute()
: error MSB4018:    at SpecFlow.Tools.MsBuild.Generation.GenerateFeatureFileCodeBehindTask.Execute()
: error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
: error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
Done building project "WebAPI.Stuff.Test.BDD.csproj" -- FAILED.

Steps to Reproduce

Must have ReSharper installed. Then

  1. Create a project for dotnet core 3.1 API
  2. Create an NUnit test project and reference API project
  3. Create a SpecFlow project to represent the acceptance test project. Referencing is immaterial
  4. Install packages for SpecFlow (SpecFlow, SpecFlow.NUnit, SpecFlow.Assist.Dynamic)
  5. Run the tests from the solution using ReSharper (Alt+R,U,N by default)

Doesn't build. Error shown: The "GenerateFeatureFileCodeBehindTask" task failed unexpectedly.

This also happens regardless of ReSharper or MSTest runner. However, when you;

  1. Remove <PackageReference Include="SpecRun.SpecFlow" Version="3.6.2" /> from the BDD .csproj file
  2. Run the tests using ReSharper, only the unit tests are shown. No SpecFlow tests are shown or even run
  3. But running the tests using MSTest work fine

Additional info:

It's possible to get it to run intermittently from this position. Though at this stage, it's not at all clear how it happens. The relevant parts of the .csproj file looks like:

<ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.5.0" />
    <PackageReference Include="NUnit" Version="3.13.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.17.0" />
    <PackageReference Include="SpecFlow.Assist.Dynamic" Version="1.4.2" />
    <PackageReference Include="SpecFlow.NUnit" Version="3.6.23" />
    <PackageReference Include="SpecFlow.Plus.LivingDocPlugin" Version="3.6.6" />

    <PackageReference Include="FluentAssertions" Version="5.10.3" />

  </ItemGroup>

Once the tests have been added to the Resharper test runner though, they work and continue to work! So there's something about that first generation of binding that something seems not to like.

Repro Project

https://www.dropbox.com/s/45bxz30sh6xis4z/WebApi.Stuff.Test.Unit.zip?dl=0

SabotageAndi commented 3 years ago

SpecRun.SpecFlow and SpecFlow.NUnit can't be installed at the same time in a SpecFlow project. They are used to configure which unit test runner is used.

I assume that you had SpecFlow+ Runner selected, when creating the SpecFlow project. This adds the SpecRun.SpecFlow NuGet package. And as you add then manually the SpecFlow.NUnit package, you are getting this error. You can select NUnit in the project wizard, to get from the start a project with NUnit.

This has nothing todo with Resharper or the .NET Core SDK and the behavior is there since SpecFlow 3.0.

Docs for Unit Test Provider: https://docs.specflow.org/projects/specflow/en/latest/Installation/Unit-Test-Providers.html