[Bug]: Console.WriteLine does not work with eventhandlers with multiple tests

[christidoc]

Version

1.44

Steps to reproduce

using Microsoft.Playwright.NUnit;

namespace WebAppUI.TestSuites;

[TestFixture]
[Parallelizable(ParallelScope.Default)]
public class MyTest : PageTest
{
    static int amountOfResponses = 0;

    [Test]
    public async Task TC1()
    {
        Page.Response += Listener;
        await Page.GotoAsync("https://playwright.dev/dotnet/");
        Console.WriteLine(amountOfResponses);
    }

    [Test]
    public async Task TC2()
    {
        Page.Response += Listener;
        await Page.GotoAsync("https://playwright.dev/dotnet/");
        Console.WriteLine(amountOfResponses);
    }

    private void Listener(object? sender, IResponse response)
    {
        Console.WriteLine(response.Url);
        amountOfResponses++;
    }
}

Expected behavior

https://playwright.dev/dotnet/ https://playwright.dev/dotnet/assets/css/styles.8a36420a.css .......... 18

https://playwright.dev/dotnet/ https://playwright.dev/dotnet/assets/css/styles.8a36420a.css .......... 36

Actual behavior

https://playwright.dev/dotnet/ https://playwright.dev/dotnet/assets/css/styles.8a36420a.css etc... 18

36

Additional context

No response

Environment

- Operating System: [Windows 11]
- Browser: [Chromium]
- .NET Version (TFM): [net8.0]

[mxschmitt]

I was able to reproduce in NUnit but not able to reproduce in MSTest, looks like a bug in NUnit.

Console.Error.WriteLine works while Console.WriteLine does not yield any output.

Filed as https://github.com/nunit/nunit/issues/4728

[CharliePoole]

Well, the NUnit framework transforms all console writes from within tests to events. The runner that handles those events gets to decide whether or not to display them as well as where to display them and how.

NUnit Console runner, for example, displays error and progress output immediately. It saves stdout output to the test result but then displays that (delayed) as well.

In the case of dotnet test, I believe the adapter gets the first crack at the events and decides what to do. I don't now how or whether Playwright gets involved with output.

Frankly, I don't think it's a bug in NUnit, unless you have a goal that NUnit should work like MsTest. That, of course, has always been the underlying assumption of the Microsoft folks.

The part that is a mystery to me is why the first message gets displayed at all. Perhaps investigate who is picking up the TextOutput event and displaying it in that case.

[mxschmitt]

Thank you for looking into this. I tried to reproduce it using NUnitLite Runner and there it surfaces the same problem: https://github.com/mxschmitt/nunit-repro-2nd-console-write-not-working/commit/160bd92a63f3044cb6dcca5955360482cf3694f7 - this means, that its no bug in dotnet test I guess.

I might try to build NUnit locally this week, if I have time in order to debug it further.