Tests are duplicated/doubled up in Visual Studio Test Explorer

[realfaisalk]

Windows 10 version - 1703, build 15063.138 Visual Studio 2017 Enterprise - 15.2 Specflow v2.20 SpecFlow.MsTest v2.2.0 SpecRun.Runner v1.6.0 SpecRun.SpecFlow v1.6.0 SpecRun.SpecFlow.2-2-0 v1.6.0

Hi,

This is not a blocker, but since upgrading SpecFlow from 2.1.0 to 2.2.0 and SpecRun 1.5.2 to 1.6.0 I've noticed that when tests are grouped by "Traits" in Test Explorer they are duplicated/doubled up (see attached screenshot). I think it may be due to having both SpecRun.SpecFlow v1.6.0 & SpecRun.SpecFlow.2-2-0 v1.6.0, but they seem to have been added by NuGet automatically. Do I need both? which is the "correct" one to have?

Thanks

[SabotageAndi]

What do you have configured as unitTestProvider in you app.config?

[realfaisalk]

Hi Andi,

Just 'SpecRun'

[PiotrZienkowicz]

I have problems with same bug:

Visual Studio 2015 Community Edition + SpecFlow For VS2015 (v2017.1.6)

[SabotageAndi]

@realfaisalk Do you have this behavior in every project or only in this one? Could you check your packages folder, if there are multiple versions of the SpecRun.Runner package.

[PiotrZienkowicz]

@SabotageAndi I have this problem in diffrent projects and only one SpecRun.Runner.1.6.0.

[realfaisalk]

@SabotageAndi

I have 2 projects in 1 solution. Both projects have this problems.

Packages:

[teknlgyuser]

@SabotageAndi

I have this problem as well.

The traits display fine until I upgrade the packages for the project to these:

package id="SpecFlow" version="2.2.0" targetFramework="net451" package id="SpecRun.Runner" version="1.6.0" targetFramework="net451" package id="SpecRun.SpecFlow" version="1.6.0" targetFramework="net451" package id="SpecRun.SpecFlow.2-2-0" version="1.6.0" targetFramework="net451"

app.config:

specFlow> unitTestProvider name="SpecRun" plugins add name="SpecRun" plugins specFlow

[SabotageAndi]

Could you try one of the examples from https://github.com/techtalk/SpecFlow.Plus.Examples and see if there you also have duplicated tests?

[realfaisalk]

Hi,

Have tried with "CustomDeploymentSteps --> CalculatorFeature". With the 'old' versions of Specflow & SpecRun the tests show fine in Test Explorer, including tags:

However when I upgrade to the latest versions then with unitTestProvider 'SpecRun' no tests are being shown in Test Explorer.

If I change unitTestProvider to 'MsTest' then they are being shown:

I get this sometimes. I have to "faff" around with the app.config file a few times before 'SpecRun' "sticks" and tests are shown in Test Explorer:

Tests are duplicated with versions 2.2.0 & 1.6.0

[SabotageAndi]

I can reproduce the behaviour and will have a look at it.

[SabotageAndi]

I found the error and I am working on a fix.

[SabotageAndi]

This is fixed and will be released with the next release of the runner.

[teknlgyuser]

Great. Thanks for working on this. When do you anticipate the next release?

[realfaisalk]

Thanks Andi,

Will test soon after the next release.

[SabotageAndi]

@teknlgyuser probably in the next weeks

[SabotageAndi]

SpecFlow+Runner 2.2.1 is released which include the fix for this issue.

[realfaisalk]

Hi,

Just got round to updating the NuGet packages, but am getting the following error when I try to use the Specflow UnitTestProvider:

[22/09/2017 09:31:52 Informational] SpecFlow + Runner Discovery started
[22/09/2017 09:31:52 Informational] SpecRun Evaluation Mode: Please purchase at http://www.specflow.org/plus to remove test execution delay.
[22/09/2017 09:31:52 Informational] Discovering target: Default
[22/09/2017 09:31:52 Informational] Discovering tests from assembly: C:\Users\XXXX\Source\Workspaces\Automation Framework\src\02 Dev\WebAutomation\bin\Debug\WebAutomation.dll
[22/09/2017 09:31:52 Informational] Error for test assembly C:\Users\XXXX\Source\Workspaces\Automation Framework\src\02 Dev\WebAutomation\bin\Debug\WebAutomation.dll
[22/09/2017 09:31:52 Informational] System.Runtime.Serialization.SerializationException: Type is not resolved for member 'TechTalk.SpecRun.Framework.Remoting.RemoteDomainInitializer,TechTalk.SpecRun.Framework.Utils, Version=1.6.0.0, Culture=neutral, PublicKeyToken=null'.
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName)
   at TechTalk.SpecRun.Framework.Remoting.AppDomainReference.CreateObject[T]()
   at TechTalk.SpecRun.Framework.Remoting.AppDomainManager.CreateAppDomainFor(String assemblyPath, Boolean useConfig, String customConfigFilePath)
   at TechTalk.SpecRun.Framework.TestSuiteProviders.TestSuiteProvider.BuildTestCollectionFromAssembly(String testAssemblyPath, FilterItem filterPredicate, INodeIdProvider& nodeIdProvider, TestCollection parentCollection)
   at TechTalk.SpecRun.Framework.TestSuiteProviders.TestSuiteProvider.GetTestSuite(TestProfile testProfile)
   at TechTalk.SpecRun.VisualStudio.TestAdapter.SpecRunTestDiscoverer.DiscoverTests(String source, IDiscoveryContext discoveryContext, IMessageLogger logger, ITestCaseDiscoverySink discoverySink)
   at TechTalk.SpecRun.VisualStudio.TestAdapter.SpecRunTestDiscoverer.DiscoverTests(IEnumerable`1 sources, IDiscoveryContext discoveryContext, IMessageLogger logger, ITestCaseDiscoverySink discoverySink)
[22/09/2017 09:31:52 Error] An exception occurred while test discoverer 'SpecRunTestDiscoverer' was loading tests. Exception: Type is not resolved for member 'TechTalk.SpecRun.Framework.Remoting.RemoteDomainInitializer,TechTalk.SpecRun.Framework.Utils, Version=1.6.0.0, Culture=neutral, PublicKeyToken=null'.
[22/09/2017 09:31:53 Informational] ========== Discover test finished: 183 found (0:00:00.5752035) ==========

Seems to have some reference to v1.6.0 stuck somewhere?

Also no tests are shown in Test Explorer if I just use Specflow, but if I use the MSTest UnitTestProvder then tests are shown.

I have followed the steps from here https://specflow.org/documentation/Troubleshooting-Visual-Studio-Integration/ but it has not fixed it

Note: I have 3 Specflow projects under the same solution.

One of the packages.config file:

<package id="ExtentReports" version="3.0.2" targetFramework="net452" />
  <package id="MSTest.TestAdapter" version="1.1.18" targetFramework="net452" />
  <package id="MSTest.TestFramework" version="1.1.18" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net452" />
  <package id="NUnit" version="3.6.1" targetFramework="net452" />
  <package id="Selenium.Firefox.WebDriver" version="0.18.0" targetFramework="net452" />
  <package id="Selenium.Support" version="3.4.0" targetFramework="net452" />
  <package id="Selenium.WebDriver" version="3.4.0" targetFramework="net452" />
  <package id="Selenium.WebDriver.ChromeDriver" version="2.31.0" targetFramework="net452" />
  <package id="Selenium.WebDriver.IEDriver" version="3.4.0" targetFramework="net452" />
  <package id="Selenium.WebDriver.MicrosoftDriver" version="15.15063.0" targetFramework="net452" />
  <package id="SpecFlow" version="2.2.1" targetFramework="net452" />
  <package id="SpecFlow.MsTest" version="2.2.1" targetFramework="net452" />
  <package id="SpecRun.Runner" version="1.6.1" targetFramework="net452" />
  <package id="SpecRun.SpecFlow" version="1.6.1" targetFramework="net452" />
  <package id="SpecRun.SpecFlow.2-2-0" version="1.6.1" targetFramework="net452" />
  <package id="System.ValueTuple" version="4.3.1" targetFramework="net452" />

[SabotageAndi]

@realfaisalk This has nothing todo with this issue. Please open a new one.

[realfaisalk]

Apologies, I thought it might be something to do with the fix from this issue, but it looks like it was just Visual Studio playing up. Had to completely remove my projects and re-download them from TFS.

**Can confirm that the duplication has gone now and tests are displayed correctly in Test Explorer.

Thanks Specflow team!**

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.