search

SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.24k stars 754 forks source link

Ignored steps fail in Test explorer when using NUnit 3.12 #2057

Closed NaZaRKIN123 closed 4 years ago

NaZaRKIN123 commented 4 years ago

SpecFlow Version:

Used Test Runner

Version number: 3.12.0

Project Format of the SpecFlow project

.feature.cs files are generated using

Visual Studio Version

Enable SpecFlowSingleFileGenerator Custom Tool option in Visual Studio extension settings

Are the latest Visual Studio updates installed?

.NET Framework:

Test Execution Method:

<SpecFlow> Section in app.config or content of specflow.json

Repro Project

Issue Description

in Specflow.Nunit package description says "Package to use SpecFlow with NUnit 3.11 and later" but when update NUnit from 3.11.0 to 3.12.0 all ignored steps are failing to find NUnit 3.11.0 dll file. Looks like it's hardcoded to be 3.11.0 but not 3.11.0+ as it states in the description.

Steps to Reproduce

  1. Install Specflow.NUnit package 3.1.95 - it will automatically install NUnit 3.11.0
  2. Update NUnit to 3.12.0
  3. Run any ignored test Actual: Test is red and error is shown. Expected: Test is yellow and ignored. Error text: Message: System.IO.FileLoadException : Could not load file or assembly 'nunit.framework, Version=3.11.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) Stack Trace: NUnitRuntimeProvider.TestIgnore(String message) TestExecutionEngine.OnAfterLastStep() TestRunner.CollectScenarioErrors() IterationSnapshotFeature.ScenarioCleanup() IterationSnapshotFeature.ChangesToIteration() line 12

Note: adding assembly redirect from this ticket resolve the issue: https://github.com/SpecFlowOSS/SpecFlow/issues/1817

SabotageAndi commented 4 years ago

You are on .NET Framework and for that, you need the binding redirects in your app.config. That's how .NET dependency works when you are not using the exact version number. For an SDK- style project they should be generated automatically. No idea why this is not happening for you.

Please provide a complete project where we can reproduce this behavior.

NaZaRKIN123 commented 4 years ago

Unfortunately, I can't provide the whole project. Wouldn't the .csproj file be enough?

<Project Sdk="Microsoft.NET.Sdk">

  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProjectGuid>{4CD37A17-F6FB-4E58-B69D-07A963217E5C}</ProjectGuid>
    <OutputType>Library</OutputType>
    <AppDesignerFolder>Properties</AppDesignerFolder>
    <RootNamespace>TestAssembly</RootNamespace>
    <AssemblyName>TestAssembly</AssemblyName>
    <TargetFramework>net462</TargetFramework>
    <FileAlignment>512</FileAlignment>
    <Configurations>Debug;Release</Configurations>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>full</DebugType>
    <Optimize>false</Optimize>
    <OutputPath>bin\Debug\</OutputPath>
    <DefineConstants>DEBUG;TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <PlatformTarget>AnyCPU</PlatformTarget>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugType>pdbonly</DebugType>
    <Optimize>true</Optimize>
    <OutputPath>bin\Release\</OutputPath>
    <DefineConstants>TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="BoDi" Version="1.4.1" />
    <PackageReference Include="Cucumber.Messages" Version="6.0.2" />
    <PackageReference Include="DotNetSeleniumExtras.WaitHelpers" Version="3.11.0" />
    <PackageReference Include="Google.Protobuf" Version="3.11.4" />
    <PackageReference Include="HtmlAgilityPack" Version="1.11.23" />
    <PackageReference Include="Newtonsoft.Json" Version="12.0.2" />
    <PackageReference Include="NUnit" Version="3.12.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.16.1">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="RestSharp" Version="106.11.3" />
    <PackageReference Include="Selenium.Support" Version="3.141.0" />
    <PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
    <PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="72.0.3626.69" />
    <PackageReference Include="Serilog" Version="2.8.0" />
    <PackageReference Include="SpecFlow" Version="3.1.95" />
    <PackageReference Include="SpecFlow.CustomPlugin" Version="3.1.95" />
    <PackageReference Include="SpecFlow.NUnit" Version="3.1.95" />
    <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.1.95" />
    <PackageReference Include="System.Configuration.ConfigurationManager" Version="4.5.0" />
    <PackageReference Include="System.ValueTuple" Version="4.5.0" />
  </ItemGroup>

  <ItemGroup>
    <None Include="API\Features\CheckVersion.feature" />
  </ItemGroup>

  <ItemGroup>
    <None Update="App.config">
      <SubType>Designer</SubType>
    </None>
  </ItemGroup>

</Project>
SabotageAndi commented 4 years ago

This csproj-file is a strange mix of sdk style and classic format. So please try to create a project where you can reproduce the issue and share this with us then.

NaZaRKIN123 commented 4 years ago

This csproj-file is a strange mix of sdk style and classic format. So please try to create a project where you can reproduce the issue and share this with us then.

This is weird as I created it from this template https://docs.specflow.org/projects/specflow/en/latest/Installation/Project-and-Item-Templates.html Used: dotnet new -i SpecFlow.Templates.DotNet && dotnet new specflowproject --unittestprovider nunit --framework net462

I'll try to provide you more info.

SabotageAndi commented 4 years ago

But this is not in our template:

<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProjectGuid>{4CD37A17-F6FB-4E58-B69D-07A963217E5C}</ProjectGuid>
    <OutputType>Library</OutputType>
    <AppDesignerFolder>Properties</AppDesignerFolder>
    <RootNamespace>TestAssembly</RootNamespace>
    <AssemblyName>TestAssembly</AssemblyName>
    <TargetFramework>net462</TargetFramework>
    <FileAlignment>512</FileAlignment>
    <Configurations>Debug;Release</Configurations>
  </PropertyGroup>
  ....

The first import is from the classic projects and not necessary for sdk- style.

And the whole Property groups for Debug and Release are normally also not needed for sdk-style projects.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.