Cannot resolve Interfaces from scenario container during run mode (debug mode works) when global container setup in Static method (BeforeTestRun)

[ts46236]

SpecFlow Version

3.9.58

Which test runner are you using?

xUnit

Test Runner Version Number

2.4.1

.NET Implementation

.NET Core 3.1

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Command line – PLEASE SPECIFY THE FULL COMMAND LINE

SpecFlow Section in app.config or content of specflow.json

{ "stepAssemblies": [ { "assembly": "xxx.xxx.xxx.SpecFlow" } ] }

Issue Description

I have a class that has both Before/AfterTestRun and Before/AfterScenario. My DI is all registered during the BeforeTestRun.

When I debug my tests thru visual studio (I'm using resharper test runner) the container resolves all dependencies. When I run the same test via command line, dotnet test or resharper's run unit test then it fails to resolve interfaces using either property injection or by calling Resolve on the container passed in during BEforeScenario (or afterScenario) or even in any downstream step.

In these failure scenarios, the IObjectContainer that gets injected (via constructor or property) cannot resolve any interfaces registered in the Global container.

The issue only arises when running the tests thru dotnet test or running them NOT in debug mode

Steps to Reproduce

Create a class with a static method that registers interfaces in the global container (passed into method decorated with BeforeTestRun).

Create a Before/AfterScenario method that depends on the interface or calls Resolve<>() on the container passed in

Error throws saying "Interface cannot be resolved"

Stack trace when resolving via method injection:

BoDi.ObjectContainerException
Interface cannot be resolved: xxx
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireScenarioEvents(HookType bindingEvent)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnScenarioStart()
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
   at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()
   at xxx.xxx.xxxFeature.ScenarioCleanup()

Stack trace when resolving via Resolve<>() on container passed into method:

BoDi.ObjectContainerException
Interface cannot be resolved: xxx
   at BoDi.ObjectContainer.TypeRegistration.<>c__DisplayClass3_0.<ResolvePerContext>b__1()
   at BoDi.ObjectContainer.RegistrationWithStrategy.ExecuteWithLock(Object lockObject, Func`1 getter, Func`1 factory, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.TypeRegistration.ResolvePerContext(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.RegistrationWithStrategy.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name)
   at BoDi.ObjectContainer.Resolve(Type typeToResolve, String name)
   at BoDi.ObjectContainer.Resolve[T](String name)
   at BoDi.ObjectContainer.Resolve[T]()
   at Cnb.Digital.Testing.Specflow.Hooks.SetUpTestAsync(IObjectContainer c) in C:\.....\Hooks.cs:line 127
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireScenarioEvents(HookType bindingEvent)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnScenarioStart()
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
   at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()
   at xxxFeature.ScenarioCleanup()

Note: resolving using the global container (set up in a static TestRun method) does resolve the interfaces correctly when run in debug or release mode.

If it works in debug mode it should work in release mode as well. Some how in debug mode the scenario level container (whose grandparent is the global container) is able to resolve services (class and interfaces) from its grandparent correctly but not in run mode.

Please help so I can clean up all the hack code I had to put in to make this work (lots of service location resolution instead of DI)

Link to Repro Project

No response

[SabotageAndi]

What you describe, looks like a bug. What is strange as I am doing the same thing quite often. Please provide a project where we can reproduce this issue. We don't have the time to find the combination to get the same behavior as you.

[michaelraue]

I could reproduce it, the special thing here is xUnit. If I switch to MSTest then it works.

[SabotageAndi]

@michaelraue can you share the project with us? Then we can have a look at it.

[michaelraue]

@SabotageAndi Alright, I have pushed a minimalistic repro project here: https://github.com/michaelraue/specflow-issue2580 In the README is a short description how you can reproduce it on your machine.

Interesting detail is that the Run/Debug difference only occurs in Jetbrains Rider (green in Debug), Visual Studio is always failing to execute the test.

[michaelraue]

I also tested NUnit, this works fine (also added it to the repro project, see comments in .csproj). So it seems to be an issue related to xUnit.

[SabotageAndi]

Ok, I found the issue. For some reason I can't remember, you don't get the global container in the BeforeTestRun hook when you request an ObjectContainer. You get a TestThreadContainer. In the case of xUnit the execution of the scenario happens on another thread then the BeforeTestRun hook and so it can't resolve the right registration.

Change it to this:

[BeforeTestRun]
public static void Startup(ObjectContainer objectContainer)
{
    var app = new WebApplicationFactory<Program>()
        .WithWebHostBuilder(builder =>
        {
        });
    objectContainer.BaseContainer.RegisterInstanceAs(app.CreateClient());
}

and it will work. This uses the global container for registration.

[ts46236]

@SabotageAndi does the parameter type have to be ObjectContainer and not IObjectContainer?

[michaelraue]

@SabotageAndi Works indeed! @ts46236 Yes it has to be, because the interface doesn't know objectContainer.BaseContainer

[michaelraue]

I tested a bit more and it works in every environment (VS, Rider, Run, Debug) with all test integration libs (MSTest, NUnit, xUnit).

@SabotageAndi would you suggest to always go with your solution, or am I generally initializing the container in a wrong way? My goal was to create the Web API and the corresponding HTTPClient just once for all scenarios, to optimize test execution performance.

Anyway thanks a lot for the fast feedback.

[SabotageAndi]

At the moment only my solution is the one that works. As said, I have no idea anymore, why we get the TestThreadContainer in the Testrun hooks if you request an ObjectContainer. I need to update the documentation with this behavior at least.

Not sure if reusing the same instance of the HTTPClient for all scenarios is the right way. Could be that you run into problems when you run your scenarios in parallel.

[R0boC0p]

Hi, I just converted my test project - which is in use for over 1 1/2 years - from Specflow.NUnit, to Specflow.xUnit, and faced this very problem, although I knew this must be a valid DI setup. Could you please mention this in the documentation, as it would have saved me quite a lot of time? Maybe here: https://docs.specflow.org/projects/specflow/en/latest/Bindings/Hooks.html#supported-hook-attributes

Thanks for you help.

edit: @SabotageAndi Unfortunately I just updated my projects which consume our main specflow library - which sets up defaults - and non of the DI is working anymore. No matter on which container or hook I try to resolve interfaces registered in the main library, I am always getting exceptions as it cannot find the registrations anymore. The main library is consumed by importing it via the specflow.json under the stepAssemblies section. This all worked flawlessly via Specflow.NUnit on the CLI using dotnet test and Visual Studio test explorer. Can you provide any help on this?

Thanks again

[SabotageAndi]

@R0boC0p Sorry, I can't help you. I don't work on SpecFlow and for Tricentis anymore.