microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
64.45k stars 3.5k forks source link

[BUG] last line of globalTeardown output is cleared by line reporter #23875

Open mxschmitt opened 1 year ago

mxschmitt commented 1 year ago

Repro:

diff --git a/examples/todomvc/globalSetup.ts b/examples/todomvc/globalSetup.ts
new file mode 100644
index 000000000..0f80bb680
--- /dev/null
+++ b/examples/todomvc/globalSetup.ts
@@ -0,0 +1,3 @@
+export default () => {
+  console.log('\\n%%from-global-setup');
+}
\ No newline at end of file
diff --git a/examples/todomvc/globalTeardown.ts b/examples/todomvc/globalTeardown.ts
new file mode 100644
index 000000000..d9faa7d89
--- /dev/null
+++ b/examples/todomvc/globalTeardown.ts
@@ -0,0 +1,3 @@
+export default () => {
+  console.log('\\n%%from-global-teardown');
+}
\ No newline at end of file
diff --git a/examples/todomvc/playwright.config.ts b/examples/todomvc/playwright.config.ts
index 06e262f2f..14fc4606d 100644
--- a/examples/todomvc/playwright.config.ts
+++ b/examples/todomvc/playwright.config.ts
@@ -46,6 +46,9 @@ export default defineConfig({
     trace: 'on-first-retry',
   },

+  globalSetup: './globalSetup.ts',
+  globalTeardown: './globalTeardown.ts',
+
   /* Configure projects for major browsers */
   projects: [
     {
@@ -57,20 +60,6 @@ export default defineConfig({
       },
     },

-    {
-      name: 'firefox',
-      use: {
-        ...devices['Desktop Firefox'],
-      },
-    },
-
-    {
-      name: 'webkit',
-      use: {
-        ...devices['Desktop Safari'],
-      },
-    },
-
     /* Test against mobile viewports. */
     // {
     //   name: 'Mobile Chrome',

inside the examples/todomvc project: npx playwright test --reporter=html.

Expected: We see the console.log of global teardown. Actual: We don't.

When setting this to true it works: https://github.com/microsoft/playwright/blob/5821c547aacfd542259c8a9707b1eee119f0b692/packages/playwright-test/src/reporters/html.ts#L63-L65

dgozman commented 1 year ago

This is because reporter (in this case, list reporter) is not aware of the terminal output produced by global teardown, and so this output is overridden with the test epilogue.

A solution would be to capture all output in the main process, but then all reporters should use a real output method instead of the captured one, which is not backwards compatible.

Running global setup/teardown in a separate process would also help, but we already do that for dependencies, so seems like not worth it.

Andarist commented 3 weeks ago

The problem is not unique to global teardown - we found out that line reporter can clear the last line of our reporter's output, in case the line reporter gets used after ours.