Visual Studio 2015 crashes when using TechTalk SpecFlow 2017.1.5

[ghost]

We have about 5 engineers using the following setup. What happens is that after a certain amount of time, the SpecFlow extension stops working. Notably, syntax coloring and code linking (Go to Definition) features stop. Then Visual Studio will be in a state where we may neither close the solution or Visual Studio and we are forced to shut it down using Task Manager. Re-opening Visual Studio fixes the problem, but only for a few hours.

We are using SpecFlow with NUnit.

Here is our setup, let me know if you need more information.

Microsoft Visual Studio Enterprise 2015
Version 14.0.25431.01 Update 3
Microsoft .NET Framework
Version 4.6.01586

Installed Version: Enterprise

Architecture and Modeling Tools   00322-80000-00000-AA148
Microsoft Architecture and Modeling Tools

UML® and Unified Modeling Language™ are trademarks or registered trademarks of the Object Management Group, Inc. in the United States and other countries.

Microsoft Visual Studio Tools for Applications 2015   00322-80000-00000-AA148
Microsoft Visual Studio Tools for Applications 2015

Visual Basic 2015   00322-80000-00000-AA148
Microsoft Visual Basic 2015

Visual C# 2015   00322-80000-00000-AA148
Microsoft Visual C# 2015

Visual C++ 2015   00322-80000-00000-AA148
Microsoft Visual C++ 2015

ASP.NET and Web Tools 2015.1   14.1.20907.0
ASP.NET and Web Tools 2015.1

ASP.NET Web Frameworks and Tools 2012.2   4.1.41102.0
For additional information, visit http://go.microsoft.com/fwlink/?LinkID=309563

ASP.NET Web Frameworks and Tools 2013   5.2.40314.0
For additional information, visit http://www.asp.net/

Azure App Service Tools v2.9.5   14.0.20810.0
Azure App Service Tools v2.9.5

Azure Data Lake Node   1.0
This package contains the Data Lake integration nodes for Server Explorer.

Azure Data Lake Tools for Visual Studio   2.1.0000.0
Microsoft Azure Data Lake Tools for Visual Studio

Common Azure Tools   1.8
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Fabric.DiagnosticEvents   1.0
Fabric Diagnostic Events

JavaScript Language Service   2.0
JavaScript Language Service

JavaScript Project System   2.0
JavaScript Project System

Microsoft .NET Core Tools (Preview 2)   14.1.20907.0
Microsoft .NET Core Tools (Preview 2)

Microsoft Azure Hive Query Language Service   2.1.0000.0
Language service for Hive query

Microsoft Azure Mobile Services Tools   1.4
Microsoft Azure Mobile Services Tools

Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2015 - v2.9.40923.2

NuGet Package Manager   3.4.4
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

PreEmptive Analytics Visualizer   1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

SlowCheetah - XML config transforms    1.0
This package enables you to transform your app.config or any other XML file based on the build configuration. It also adds additional tooling to help you create XML transforms.

SQL Server Analysis Services   13.0.1701.8
Microsoft SQL Server Analysis Services Designer 
Version 13.0.1701.8

SQL Server Data Tools   14.0.61021.0
Microsoft SQL Server Data Tools

SQL Server Integration Services   
Microsoft SQL Server Integration Services Designer
Version 13.0.1601.5

SQL Server Reporting Services   13.0.1701.8
Microsoft SQL Server Reporting Services Designers 
Version 13.0.1701.8

TechTalk SpecFlow   30F08A29-D27E-42FF-92D3-50391313A1EF
TechTalk SpecFlow - Binding business requirements to .NET code, http://www.specflow.org
Copyright © 2009–2017 TechTalk

ToolWindowHostedEditor   1.0
Hosting json editor into a tool window

TypeScript   1.8.36.0
TypeScript tools for Visual Studio

[ghost]

A little bit more info:

• About 200 individual Scenarios
• 40 Feature files = about 6,000 lines of C#
• 42 Step files = about 7,000 lines of C#

The symptom that seems to grow is that the SpecFlow Step Matching results in an error dialog:

"Step bindings are still being analyzed. Please wait."

Now upon opening the project this takes approximately 30 seconds before the step bindings are analyzed. What I am guessing is happening is that eventually this is timing out because of other things going in Visual Studio and then deadlocks, crashing Visual Studio completely.

Question is: do you think we have too many features and steps in one project?

[SabotageAndi]

@jmb-mage The size of your project is no problem. I have a project with about 9000 scenarios and a lot of more stepfiles.

The Visual Studio Integration caches some data in the %TEMP% directory with name specflow-stepmap-*.cache. Try to delete the files. After opening the solution again, this caches should be regenerated.

Additionally you can enable tracing in the options:

You get than a new output with tracing information:

Perhaps you see there some exception or other clues.

[ghost]

I enabled the SpecFlow output tracing. The most recent entries are:

[13:28:52.6804528] VsProjectFileTracker: Build Done.
[13:28:52.7414528] StepMap: StepMap with 53 feature files and 748 step definitions saved
[13:28:54.9987681] RemoteAppDomainTestGeneratorFactory: AppDomain for generator disposed

At which point the Generator was apparently disposed. I was in Visual Studio at the time, looking at a feature file which went from the beautiful syntax coloring to a bunch of bold text as that happened.

Now pressing F12 on a step yields this error:

[SabotageAndi]

@jmb-mage Is this error happening for you in every project with SpecFlow? Does it also happen if you open an example from us? They are here: https://github.com/techtalk/SpecFlow.Plus.Examples

If not, could you provide more information about your solution and the configuration of SpecFlow?

[ghost]

The https://github.com/techtalk/SpecFlow.Plus.Examples worked with SpecFlow Bindings no problem.

What kind of information from the Solution do you want?

I provided some in the post above. We are are now up to 244 tests 49 Feature files - 10,248 lines of C# 50 Step files - 10,039 lines of C# 24 Page files - 3,729 lines of C#

Settings:

[SabotageAndi]

@jmb-mage I was more thinking about things like:

• If you have the steps and features in the same project
• If you made any changes to the specflow section in the app.config
• Do you have multiple projects with SpecFlow? Do they all use the same version?

Could you compare the examples with your project. Is there something big different (except the size of the project)?

[ghost]

•If you have the steps and features in the same project YES

•If you made any changes to the specflow section in the app.config

  <specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
    <unitTestProvider name="NUnit" />
  </specFlow>

•Do you have multiple projects with SpecFlow? Do they all use the same version? YES TEST & STEPS Project:

  <package id="SpecFlow" version="2.1.0" targetFramework="net461" />
  <package id="SpecFlow.NUnit" version="2.1.0" targetFramework="net461" />

PAGES Project

  <package id="SpecFlow" version="2.1.0" targetFramework="net452" />

EDIT: just noticed the targetFramework is different in the two projects, is that a concern?

[SabotageAndi]

Ok, there are 2 places where we throw the exception

https://github.com/techtalk/SpecFlow.VisualStudio/blob/master/VsIntegration/LanguageService/GherkinLanguageService.cs#L113 https://github.com/techtalk/SpecFlow.VisualStudio/blob/master/VsIntegration/LanguageService/GherkinLanguageService.cs#L251

It could be that this is really a timing issue. I will try to create next week a big project and try to reproduce it.

@jmb-mage: How many Scenarios and Bindings (aka Given/When/Then- Attributes) do you have in your solution approximately? I would need a number and not lines of code.

[ghost]

TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo was found 244 times in 42 files GivenAttribute has 198 references WhenAttribute has 219 references ThenAttribute has 406 references

There are some scenarios and bindings that are commented out, that's why the numbers are a little off.

[ghost]

One of our Team Leads came up with this fix, inside the project .csproj:

  <Import Project="..\packages\SpecFlow.2.1.0\tools\TechTalk.SpecFlow.targets" Condition="Exists('..\packages\SpecFlow.2.1.0\tools\TechTalk.SpecFlow.targets')" />
  <!--<ItemGroup>
    -->
  <!-- include all feature files from the folder "FeatureFiles" -->
  <!--
    <None Include="Features\**\*.feature" /> 
  </ItemGroup>
  <Target Name="AfterUpdateFeatureFilesInProject">
      -->
  <!-- include any files that specflow generated into the compilation of the project -->
  <!--
      <ItemGroup>
          <Compile Include="@(SpecFlowGeneratedFiles)" />
      </ItemGroup>
  </Target>-->

What that does is include all the .feature files in the build, forcing it to re-generate the code behind .feature.cs files. But it changes all those files, which makes the repository think they've been updated.

However, this does fix the SpecFlow Bindings in our Solution.

[ghost]

After doing that monkey patch <Import Project="..\packages\SpecFlow.2.1.0\tools\TechTalk.SpecFlow.targets" Condition="Exists('..\packages\SpecFlow.2.1.0\tools\TechTalk.SpecFlow.targets')" /> in the .csproj for a few months we eventually found out about Scoped-Bindings. Now are using it in every feature and step file.

1.In every Feature file, at the top declare a tag that will name the steps file.
Ex: UserPage.feature will have
@userPageSteps as a tag at the top

2. The tag will be used in the step file(s) invoked by the .Feature
   Ex: above class declaration in UserPageSteps.cs add
   [Binding, Scope(Tag='userPageSteps')]

[SabotageAndi]

We don't support VS2015 anymore => closed

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.