SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.22k stars 751 forks source link

Missing stacktrace when throwing exceptions in async hook #2716

Open FantasyTeddy opened 11 months ago

FantasyTeddy commented 11 months ago

SpecFlow Version

3.7.13 - 3.9.74

Which test runner are you using?

NUnit

Test Runner Version Number

3.7.13 - 3.9.74

.NET Implementation

.NET 6.0

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Visual Studio Test Explorer

SpecFlow Section in app.config or content of specflow.json

No response

Issue Description

I have not found any definitive answer if asynchronous hooks are even fully supported yet.

However, I found that throwing exceptions inside a hook does not report a useful stacktrace, instead it produces the following:

System.NotImplementedException : The method or operation is not implemented.
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireScenarioEvents(HookType bindingEvent)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnStepEnd()
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(IContextManager contextManager, StepInstance stepInstance)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.Step(StepDefinitionKeyword stepDefinitionKeyword, String keyword, String text, String multilineTextArg, Table tableArg)
   at TechTalk.SpecFlow.TestRunner.Given(String text, String multilineTextArg, Table tableArg, String keyword)
   at SpecFlowProject1.Features.CalculatorFeature.AddTwoNumbers() in D:\Projects\SpecFlowProject1\SpecFlowProject1\Features\Calculator.feature:line 4

Before version 3.7.13 the same code produced the much more useful output:

System.NotImplementedException : The method or operation is not implemented.
   at SpecFlowProject1.Hooks.Hooks.AfterStepHook() in D:\Projects\SpecFlowProject1\SpecFlowProject1\Hooks\Hook.cs:line 20
   at TechTalk.SpecFlow.Bindings.SynchronousBindingDelegateInvoker.<>c__DisplayClass2_0.<<InvokeBindingDelegateAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at TechTalk.SpecFlow.Bindings.AsyncHelpers.<>c__DisplayClass1_0`1.<<RunSync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at TechTalk.SpecFlow.Bindings.AsyncHelpers.ExclusiveSynchronizationContext.BeginMessageLoop()
   at TechTalk.SpecFlow.Bindings.AsyncHelpers.RunSync[T](Func`1 task)
   at TechTalk.SpecFlow.Bindings.SynchronousBindingDelegateInvoker.InvokeBindingDelegateAsync(Delegate bindingDelegate, Object[] invokeArgs)
   at TechTalk.SpecFlow.Bindings.SynchronousBindingDelegateInvoker.InvokeDelegateSynchronously(Delegate bindingDelegate, Object[] invokeArgs)
   at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireScenarioEvents(HookType bindingEvent)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnStepEnd()
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(IContextManager contextManager, StepInstance stepInstance)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.Step(StepDefinitionKeyword stepDefinitionKeyword, String keyword, String text, String multilineTextArg, Table tableArg)
   at TechTalk.SpecFlow.TestRunner.Given(String text, String multilineTextArg, Table tableArg, String keyword)
   at SpecFlowProject1.Features.CalculatorFeature.AddTwoNumbers() in D:\Projects\SpecFlowProject1\SpecFlowProject1\Features\Calculator.feature:line 4

Steps to Reproduce

The behavior can be observed by throwing an exception e.g. in the AfterStep hook:

[AfterStep]
public async Task AfterStepHook()
{
    await Task.Delay(1000);

    throw new NotImplementedException();
}

Link to Repro Project

No response