SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.23k stars 752 forks source link

Scoped AfterScenario running outside scope #2558

Open 0lks opened 2 years ago

0lks commented 2 years ago

SpecFlow Version

3.9.22 & 3.9.50

Which test runner are you using?

SpecFlow+ Runner

Test Runner Version Number

3.9.31

.NET Implementation

.NET 5.0

Project Format of the SpecFlow project

Classic project format using <PackageReference> tags

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Visual Studio Test Explorer

SpecFlow Section in app.config or content of specflow.json

No response

Issue Description

Here is the bare-bones structure of the test I'm working with

Feature file

Feature: FeatureX

@SomeTag
Scenario: ScenarioX

Step Definition file

[Binding, Scope(Tag = "SomeTag")]
public class STEPS
{
    ... Step definitions...

    [AfterScenario("Teardown_SomeTag")
    public void Teardown() {...}
}

In the above example ScenarioX executes steps in the definition class STEPS. In it, AfterScenario hook should run only when the scenario is also marked with the tag @Teardown_SomeTag. My scenario is not marked with that tag, yet the hook is executed regardless.

If I remove the scope 'SomeTag' from the class and from the scenario, the hook will behave normally and not fire off unexpectedly when the scenario does not have the required tag. However I need those scopes to be able to reuse simple step definition wordings that run slightly differently behind the scenes, depending on the state of the AUT. Without them the feature file readability suffers.

Steps to Reproduce

See the simple repro project linked - run both scenarios and observe how the scenario without the hook scoping tag still executes the hook.

Link to Repro Project

https://github.com/0lks/HookScopeIssueDemo

singh-swati28 commented 8 months ago

@SabotageAndi , any update on the Scoped hook running outside scope. I am also facing same issue with Specflow + Xunit (3.9.74) version. It was working fine with SpecRunner but seems broken for Xunit test runner

I tried specifying tag in following ways but neither seems working correctly:

[AfterScenario("scenario_specific_tag")] public void ResetPreferences() { Utilities.ResetData(preferences); }

[AfterScenario] [Scope(Tag = "scenario_specific_tag")] public void ResetPreferences() { Utilities.ResetData(preferences); }

On executing scenarios that are not tagged with 'scenario_specific_tag', AfterScenario hook is running and failing thowing Null Reference exception for variable preferences.

If, I add a null check for preferences variable, in that case AfterScenario hook is not getting executed for out of scope tests

[AfterScenario] [Scope(Tag = "scenario_specific_tag")] public void ResetPreferences() { if(!string.IsNullOrWhitespace(preferences)){ Utilities.ResetData(preferences); } }

Could you please update what is the correct way of using Scoped Hooks?

Regards, Swati

SabotageAndi commented 8 months ago

@singh-swati28 I don't work for Tricentis and on SpecFlow anymore. Sorry I can't help you.