Early Termination of ScenarioContext in Parallel Execution

[jcomish]

SpecFlow Version:

• [x] 2.2
• [ ] 2.1
• [ ] 2.0
• [ ] 1.9

Used Test Runner

• [ ] SpecFlow+Runner
• [ ] MSTest
• [x] NUnit
• [ ] Xunit

Version number: 3.9.0

Visual Studio Version

• [x] VS 2017
• [ ] VS 2015
• [ ] VS 2013

Are the latest Visual Studio updates installed?

• [x] Yes
• [ ] No

.NET Framework:

• [x] >= .NET 4
• [ ] before .NET 4

Test Execution Method:

• [x] Visual Studio Test Explorer
• [ ] TFS/VSTS – Task – PLEASE SPECIFY THE NAME OF THE TASK
• [ ] Command line – PLEASE SPECIFY THE FULL COMMAND LINE

<SpecFlow> Section in app.config

<specFlow>
    <unitTestProvider name="NUnit" />
  </specFlow>

Issue Description

I am running into an issue with what I believe is the ScenarioContext. I am trying to run scenarios concurrently. I am building a framework that spins up a user-specified amount of browser nodes to do this. The test workers spawned by NUnit then finds an available browser node to run on and executes the test.
When running non-Specflow tests with the framework, it works like a charm. I am consistently running into an error however, with the Specflow tests.

The Error When I am running more than one test worker, my last Gherkin scenario fails every time with the following error:

System.NullReferenceException : Object reference not set to an instance of an object.
TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.GetHookContainer(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at Project.Web.AcceptanceTests.Tests.SpecFlow.ReviewResources.Features.UserClicksReviewerResources_ReviewResourcesFeature.ScenarioSetup(ScenarioInfo scenarioInfo)
   at Project.Web.AcceptanceTests.Tests.SpecFlow.ReviewResources.Features.UserClicksReviewerResources_ReviewResourcesFeature.UserAttemptsToClicksReviewerResources_ReviewResourcesWithVariousInputParameters(String clicks_Navigation, String clicks_Trigger, String status, String[] exampleTags) in D:\DEV\TestAutomation\TestAutomationFramework\Test\FFESpecflow\Analyzer\Tests\SpecFlow\ReviewResources\Features\ReviewResources_ReviewerResources.feature:line 7
--TearDown
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.GetHookContainer(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnScenarioEnd()
   at Project.Tests.SpecFlow.Feature.ScenarioTearDown()

It appears that when the last two threads are running, the one that finishes first will teardown the ScenarioContext and the last thread fails because the ScenarioContext does not exist. I have verified this by noticing that the last thread throws an exception before the "AfterFeature" method is reached. A hacky workaround further confirms this theory. The tests work if I have each thread wait in the teardown while the other threads finish their tests. This kind of defeats the purpose of the parallelization though.

I couldn't find any issues that matched this exactly... Is this a known issue? Any ideas on a workaround?

Steps to Reproduce

• Set nunit parallelization to children
• Set the Level of Parallelization to 2 or more.

[SabotageAndi]

@gasparnagy Any ideas?

[jcomish]

I currently have a seemingly functional workaroud... On each teardown, I am counting the number of "processing" threads (this status is set at the beginning of each scenario setup). If the number is outside of a certain threshold, I wait until the number of running threads comes down (indicating that the tests are running out). After that is complete, I resync my threads by waiting until the last thread is completely done, then move on to the next test. The tests are thus being completed in groups of n, where n is the level of parallelization.

While this approach works, there is some thread downtime while there are no tests to send to it. Guidance on being able to run the scenarios concurrently without the scenariocontext being torn down prematurely would be helpful.

[jcomish]

Looks like this is a duplicate of #894

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.