No way to distinguish between individual Scenarios in a Scenario Outline - ScenarioInfo.Title not Correct

alistairjevans commented 4 years ago

SpecFlow Version:

[x] 3.0
[ ] 2.4
[ ] 2.3
[ ] 2.2
[ ] 2.1
[ ] 2.0
[ ] 1.9

Used Test Runner

[x] SpecFlow+Runner
[ ] MSTest
[ ] NUnit
[ ] Xunit

SpecRun.Runner 3.0.391

Project Format of the SpecFlow project

[ ] Classic project format using packages.config
[x] Classic project format using <PackageReference> tags
[ ] Sdk-style project format

.feature.cs files are generated using

[x] SpecFlow.Tools.MsBuild.Generation NuGet package
[ ] SpecFlowSingleFileGenerator custom tool

Visual Studio Version

[ ] VS 2019
[x] VS 2017
[ ] VS 2015

`Enable SpecFlowSingleFileGenerator Custom Tool` option in Visual Studio extension settings

[ ] Enabled
[x] Disabled

Are the latest Visual Studio updates installed?

[x] Yes
[ ] No, I use Visual Studio version <Major>.<Minor>.<Patch>

.NET Framework:

[x] >= .NET 4.5
[ ] before .NET 4.5
[ ] .NET Core 2.0
[ ] .NET Core 2.1
[ ] .NET Core 2.2
[ ] .NET Core 3.0

Test Execution Method:

[x] Visual Studio Test Explorer
[ ] TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
[ ] Command line – PLEASE SPECIFY THE FULL COMMAND LINE

Issue Description

In our tests, we have to allocate a test database at the start of each feature, which is shared by all scenarios in the feature.

If a scenario fails, then because the data may be in a half state, we throw away the database so the next scenario gets a fresh one.

When a scenario is configured to auto-retry, we look at the ScenarioInfo.Title in BeforeScenario and compare it to a value we placed in the FeatureContext to know if we're in a retry, so we can clean out the database in each retry.

However, when a Scenario Outline runs, all the individual Examples have the same value for ScenarioInfo.Title, so I can't distinguish between them.

This is because, in the generated code, while the ScenarioAttribute on the test method has the correct title, e.g.

[TechTalk.SpecRun.ScenarioAttribute("Scenario Outline Name, Test Case 1", SourceLine=525)]

The ScenarioInfo block used to initialise the Scenario Context does not use this value:

TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo("Scenario Outline Name", null, exampleTags);
#line 69
this.ScenarioInitialize(scenarioInfo);
this.ScenarioStart();

So I have no way to distinguish in code between the two scenarios, so I think they are all retries, and I fetch new databases for no reason.

Would it be possible to update the ScenarioInfo constructor in the generated method to take a parameter through from the caller that contains the actual title?

SabotageAndi commented 3 years ago

See https://github.com/SpecFlowOSS/SpecFlow/issues/1890#issuecomment-631324071 for answer

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

SpecFlowOSS / SpecFlow