Tests pass locally but fail from GitHub actions

[saliej]

SpecFlow Version

4.0.16-beta

Which test runner are you using?

xUnit

Test Runner Version Number

4.0.16-beta

.NET Implementation

.NET 6.0

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Command line – PLEASE SPECIFY THE FULL COMMAND LINE

SpecFlow Section in app.config or content of specflow.json

No response

Issue Description

Command executed by Github action: dotnet test MyTestProject.IntegrationTests --no-build --verbosity normal --logger "console;verbosity=detailed"

I suspect this problem is a timing issue related this bug: https://github.com/SpecFlowOSS/SpecFlow/issues/2608

... because in the Github action log I observe the following errors:

[xUnit.net 00:00:33.93]     Get GSF Thresholds [FAIL]
[xUnit.net 00:00:33.94]       System.ObjectDisposedException : Cannot access a disposed object.
[xUnit.net 00:00:33.94]       Object name: 'IServiceProvider'.
[xUnit.net 00:00:33.94]       Stack Trace:
[xUnit.net 00:00:33.94]            at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
[xUnit.net 00:00:33.94]            at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
[xUnit.net 00:00:33.94]            at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
[xUnit.net 00:00:33.94]            at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
[xUnit.net 00:00:33.94]            at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.CreateScope(IServiceProvider provider)
*** [xUnit.net 00:00:33.94]         /home/runner/work/MyTestProject/MyTestProject/MyTestProject.IntegrationTests/SpecFlow/Steps/CommonSteps.cs(92,0): at MyTestProject.IntegrationTests.SpecFlow.Steps.CommonSteps.GivenTheImmutableThresholds(Table table)
[xUnit.net 00:00:33.94]            at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[xUnit.net 00:00:33.94]            at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
[xUnit.net 00:00:33.94]            at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBindingAsync(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, DurationHolder durationHolder)
[xUnit.net 00:00:33.94]            at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatchAsync(BindingMatch match, Object[] arguments, DurationHolder durationHolder)
[xUnit.net 00:00:33.94]            at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepAsync(IContextManager contextManager, StepInstance stepInstance)
[xUnit.net 00:00:33.94]            at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStepAsync()
[xUnit.net 00:00:33.94]            at TechTalk.SpecFlow.TestRunner.CollectScenarioErrorsAsync()
***  [xUnit.net 00:00:33.94]            at MyTestProject.IntegrationTests.SpecFlow.Features.GetThresholdsFeature.ScenarioCleanupAsync()
[xUnit.net 00:00:33.94]         /home/runner/work/MyTestProject/MyTestProject/MyTestProject.IntegrationTests/SpecFlow/Features/GetThresholds.feature(16,0): at MyTestProject.IntegrationTests.SpecFlow.Features.GetThresholdsFeature.GetGSFThresholds()
[xUnit.net 00:00:33.94]         --- End of stack trace from previous location ---

The lines prefixed by *** are the Feature Background and ScenarioCleanup methods respectively. It seems like the Cleanup is occurring before the Background method has completed.

The generated code looks like this:

                await this.ScenarioStartAsync();
#line 3
this.FeatureBackgroundAsync(); // BUG #2608
#line hidden
#line 15
 await testRunner.WhenAsync("I call the API with the values \"FOO\"", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "When ");
#line hidden
                // Generated table definition omitted
#line 16
 await testRunner.ThenAsync("I should receive the following Thresholds", ((string)(null)), table16, "Then ");
#line hidden
            }
            await this.ScenarioCleanupAsync();

The related bug is after #line 3 (the call is not awaited).

Is there a suitable workaround for this problem, or should I just wait for the next release that includes the fix?

Steps to Reproduce

Execute a Specflow test that disposes required services via a Github action using Specflow 4.0.16-beta

Link to Repro Project

No response

[saliej]

I've decided to use the CI Nuget Feed until the fix officially released.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.