Test event hooks [AfterTestRun] , and Nunit [OneTimeTearDown] run before the TestExecution.json is generated on VS for mac with .NET 6

Moshex commented 1 year ago

SpecFlow Version

3.9.74

Which test runner are you using?

NUnit

Test Runner Version Number

3.13.3

.NET Implementation

.NET 6.0

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlowSingleFileGenerator custom tool

Test Execution Method

Visual Studio Test Explorer

SpecFlow Section in app.config or content of specflow.json

No response

Issue Description

I have a project using .net framework version 4.8 and in that code base, I use a code that generates the living doc vai the code with the console command. in .net framework version 4.8 the TestExecution.json would be generated after the nunit hook of [OneTimeTearDown] allowing the code to generate an updated living doc based on the tests that have been just run. However, with .NET 6, the TestExecution.json is now being generated after the test event hook of [OneTimeTearDown]. This makes it to that it's generating a report off of the previous run and not the run that has just been executed. is there a way to invoke via C# code to generate the TestExecution.json when I choose to do so? or is there any other idea out there that I can use to work around this issue? Keep in mind I can't use the hook of [AfterTestRun] since that hook does not work on mac. Thank you for any help in advance!

Steps to Reproduce

Have a mac mobile automation project .NET 6 and specflow

make a method in the code to generate a living doc that looks like this:

public static void GenerateLivingDoc()
    {
        var getAppType = GlobalContext.scenarioContext?.Get<Models.GetAppType>();
        var appType = @" --Standard-- Application";

        var ReportTitle = "Mobile Automation " + appType
            + ": {Environment = " + AppInitializer.CurrentBranch + "}";

        var workingDirectory = Directory.GetCurrentDirectory();
        var currentDir = Directory.GetParent(workingDirectory)?.FullName;
        var projectDir = Directory.GetParent(currentDir!)?.Parent?.FullName; ;
        var command = @" feature-folder " + @projectDir + @"/ -t " + @workingDirectory
            + "/TestExecution.json --project-name \"HEB Mobile Automation\" --title \""
            + @ReportTitle + "\" --output " + @workingDirectory + @"/LivingDoc.html";
        try
        {
            Process proc = new Process();
            proc.StartInfo.FileName = "/Users/"+AppInitializer.Username+"/.dotnet/tools/livingdoc";
            proc.StartInfo.Arguments = command;
            proc.StartInfo.RedirectStandardError = true;
            proc.StartInfo.RedirectStandardOutput = true;
            proc.StartInfo.CreateNoWindow = true;
            proc.StartInfo.UseShellExecute = false;
            proc.Start();
            string mainOutPut = proc.StandardOutput.ReadToEnd();
            string errorOuput = proc.StandardError.ReadToEnd();

            Console.WriteLine("Standard Out put: " + mainOutPut + System.Environment.NewLine +
                "Error Output: " + errorOuput + System.Environment.NewLine + proc.ExitCode );
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
        }
    }

Place that method in a void with the hooks of [AfterTestRun], and Nunit [OneTimeTearDown]
Run the tests
Observe how the report is generated

Link to Repro Project

No response

goblinmaks commented 4 months ago

You can generate doc by livingdoc tool or by tool from nuget package, i think it is specially developed for your purposes.

Moshex commented 4 months ago

@goblinmaks
do you have a tool in mind that works with C# ?

goblinmaks commented 3 months ago

https://docs.specflow.org/projects/specflow-livingdoc/en/latest/LivingDocGenerator/Setup-the-LivingDocPlugin.html

aw2003 commented 2 months ago

I'm not sure anyone understood the question yet - I'm trying to do the same as @Moshex, which is to call the "livingdoc" CLI tool inside the [AfterTestRun] hook - that's what his code does, it formats up the command line to call the tool.

I'm trying to do this in .Net 7.0 on Windows, rather than Mac, but the idea is the same - I suspect Windows will show the same behaviour (not checked yet) and TestExecution.json won't be available to be processed until after the [AfterTestRun] hook is executed. I'm also running using NUnit.

@Moshex - have I got this right?

Moshex commented 2 months ago

@aw2003 you are correct, I am also happy to share a solution that I have found please take a look at this link and let me know if you have any questions on how to implement the fix. https://stackoverflow.com/a/78083993/6421277

SpecFlowOSS / SpecFlow