[BeforeTestRun] and [AfterTestRun] blocks runs for every Test cases in single execution cycle on MAC - Safari

[shubautomation]

Hi @gasparnagy and @SabotageAndi

I am not able to understand how the workaround mentioned by you in step 3 will work for [AfterTestRun] as we dont know how many test cases will run at one time. So what will be the condition to check that [AfterTestRun] should execute only once that too at the end of execution cycle. Please help me here, it is blocker for me. Kindly suggest

@shubautomation I have reviewed the solution and this looks good. I don't know why you got the problems on Mac (I don't have a Mac, so cannot test it unfortunately).

My suggestions:

• Check what happens if you run the tests on Mac from the command line and not from Visual Studio (dotnet test)
• Check what happens if you use an other IDE, like Visual Studio for Mac 2022 or Visual Studio Code
• If nothing above helps, you can still workaround the problem, by having a static private bool wasBeforeTestRunExecuted field in Hooks.cs and at the first line of the [BeforeTestRun] hook write: if (wasBeforeTestRunExecuted) return; wasBeforeTestRunExecuted = true; and the same for the [AfterTestRun].

I hope this helps.

Originally posted by @gasparnagy in https://github.com/SpecFlowOSS/SpecFlow/issues/2652#issuecomment-1285272005

[shubautomation]

Hi @gasparnagy and @SabotageAndi

I am not able to understand how the workaround mentioned by you in step 3 will work for [AfterTestRun] as we dont know how many test cases will run at one time. So what will be the condition to check that [AfterTestRun] should execute only once at too at the end of execution cycle. Please help me here, it is blocker for me. Kindly suggest

@shubautomation I have reviewed the solution and this looks good. I don't know why you got the problems on Mac (I don't have a Mac, so cannot test it unfortunately).

My suggestions:

• Check what happens if you run the tests on Mac from the command line and not from Visual Studio (dotnet test)
• Check what happens if you use an other IDE, like Visual Studio for Mac 2022 or Visual Studio Code
• If nothing above helps, you can still workaround the problem, by having a static private bool wasBeforeTestRunExecuted field in Hooks.cs and at the first line of the [BeforeTestRun] hook write: if (wasBeforeTestRunExecuted) return; wasBeforeTestRunExecuted = true; and the same for the [AfterTestRun].

I hope this helps.

Originally posted by @gasparnagy in #2652 (comment)

@gasparnagy

Could you please help me here?

[shubautomation]

Hi @gasparnagy @SabotageAndi

Could you please help me here. In MAC [BeforeTestRun] and [AfterTestRun] blocks runs for every Test cases in single execution cycle.

[gasparnagy]

I think you should just follow my suggestion. It uses a static variable to remember if the hook was executed, so it will not execute for the second test regardless of the number of tests.

[shubautomation]

I think you should just follow my suggestion. It uses a static variable to remember if the hook was executed, so it will not execute for the second test regardless of the number of tests.

Hi @gasparnagy

Thanks for the reply.

I follow your suggestion and got query for [AfterTestRun]. We can have Static variable in [BeforeTestRun] and it will not execute the block second time regardless of Test cases. But for [AfterTestRun] block, how will be knowning the last test case is executed (as we dont know how many test cases will run) and then [AfterTestRun] block should run.

Example: Lets say I run 5 Test cases in one execution cycle, then after last test cases gets executed then only [AfterTestRun] block should get execute. But we dont know the count of the test cases so that we can make [AfterTestRun] block executed only after last test case gets executed. Static variable alone will not serve the purpose I believe.

Note: In Mac-Safari, [BeforeTestRun] and [AfterTestRun] gets executed after each test case. ideally it should get executed only once.

Could you please explain in detail for [AfterTestRun] block. Thanks in advance

[gasparnagy]

@shubautomation I see. The [AfterTestRun] gets executed from an NUnit hook ([OneTimeTearDown]), so if that is executed multiple times than that is most probably an NUnit bug. I don't think you can have a workaround for that within SpecFlow, because - as you noticed - you will never know if there will be additional tests coming after that. Maybe switching to MsTest or xUnit would work.

So I would probably submit the issue for the NUnit team. As a reference this is the code SpecFlow includes to the project that uses the NUnit hooks:

using System.CodeDom.Compiler;
using System.Diagnostics;
using global::NUnit.Framework;
using global::TechTalk.SpecFlow;
using global::System.Runtime.CompilerServices;

[GeneratedCode("SpecFlow", "3.9.74")]
[SetUpFixture]
public class MyCalculator_Specs_NUnitAssemblyHooks
{
    [OneTimeSetUp]
    [MethodImpl(MethodImplOptions.NoInlining)]
    public void AssemblyInitialize()
    {
        var currentAssembly = typeof(MyCalculator_Specs_NUnitAssemblyHooks).Assembly;

        TestRunnerManager.OnTestRunStart(currentAssembly);
    }

    [OneTimeTearDown]
    [MethodImpl(MethodImplOptions.NoInlining)]
    public void AssemblyCleanup()
    {
        var currentAssembly = typeof(MyCalculator_Specs_NUnitAssemblyHooks).Assembly;

        TestRunnerManager.OnTestRunEnd(currentAssembly);
    }
}

I close this issue for now, please reopen if you have further input.

[shubautomation]

Hi @gasparnagy

Thanks for the reply. I have raised the issue to nUnit. Hope they are to able to fix the issue or provide some workaround

Btw can you please let me know what is the command to run the tests from Dot net command line in Mac OS ? Want to try from the dot net command and see if it is making any difference

[gasparnagy]

@shubautomation I have never used .NET on Mac, but I think dotnet test should work everywhere.

Feel free to paste the link to the NUnit issue here, so that it will be easier to track.

[shubautomation]

Hi @gasparnagy

ohh so dotnet test is the only command to execute specflow tests. And from where we should execute it ? from which folder?

nUnit issue link : https://github.com/nunit/nunit/issues/4275

[gasparnagy]

from a test project folder (where the .csproj is). more info: https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test

[shubautomation]

@gasparnagy

Thanks a lot for the information. I Hope I should get some workable solution for my issue. Finger crossed.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.