[Feature] Support @playwright/test in electron

[sumitnegi7]

We are using playwright to develop web app and electron application. In web apps it works pretty well but in electron some features work some does not like multiple project support does not work, code coverage does not work. It would be really nice if playwright officially supports electron applications also.

[dbjorge]

Our team is very interested in this as well (for Accessibility Insights for Android, which is an electron app despite the name).

This is likely to become a bigger market gap to fill in the near future because the leading competitor to Playwright for Electron, Spectron, has an open issue tracking how they have no active maintainers and will break with Electron v14+. Because of this, our team would really like to prioritize a migration off of Spectron (we're probably going to do that even if we have to run Playwright using a different runner, but it would be great to be able to switch to @playwright/test as we're migrating anyway)

[pavelfeldman]

Despite the name, @playwright/test is a general purpose test runner. What makes it a Playwright runner is a single 200 line file that defines all the useful fixtures that are passed into the test. So adding support for Electron would mean adding a probably smaller fixture file that covers your scenarios.

You could either do it in a PR, or you could tell us what fixtures you'd like to have. I would imagine

• you'd want to specify the app in the project config
• get the electronApp fixture in each test
• have a mode where each test gets a fresh app and a mode where you start once and all tests reuse the app
• get a trace/video for all the browser windows during the test run

Is there more? Those above are really trivial...

[sumitnegi7]

@pavelfeldman the first point you mentioned you'd want to specify the app in the project config already work just the multiple project config setting is not working properly.

Example:

  projects: [
    {
      name: "Desktop Screen",
      use: {
        viewport: devices["Desktop Chrome"].viewport,
      },
    },
{
      name: "Desktop Screen",
      use: {
        viewport: devices["Desktop Chrome"].viewport,
      },
    },
{
      name: "iPhone Screen",
      use: {
        viewport: devices["iPhone 12"].viewport,
      },
    },

{
      name: "iPadScreen",
      use: {
        viewport: devices["iPad Pro 11"].viewport,
      },
    },
  ],

Currently when i take screenshot using the above projects the screenshots are same for all project and can you hide the screen/window like we do in the case of browsers by setting {headless:true}.

[pavelfeldman]

@sumitnegi7 I think we hit some misunderstanding here. I thought this was about using @playwright/test for testing Electron. I think @dbjorge is on the same page with me here. In the scope of this, I don't understand your last comment.

[shadow-light]

Anyone looking to do this, it's been implemented in Playwright's own tests already: https://github.com/microsoft/playwright/tree/master/tests/electron

More integrated support with the CLI etc would be great though

[sumitnegi7]

@sumitnegi7 I think we hit some misunderstanding here. I thought this was about using @playwright/test for testing Electron. I think @dbjorge is on the same page with me here. In the scope of this, I don't understand your last comment.

@pavelfeldman i meant that it is not working in headless mode it always launches the election application window.

[UberMouse]

Does Electron support a headless mode via a CLI flag like browsers? I thought not. Unless you're creating multiple BrowserWindows in multiple places in the main code it's trivial to implement headless support yourself, just slap show: false on the BrowserWindow options.

[UberMouse]

I've cobbled together an @playwright/test fixture/config that works with Electron from looking at the Electron tests in this repository https://gist.github.com/UberMouse/facbe751c3ecb9b31e8b4f6221567b7a

However, I can't get it to capture traces automatically. I'm guessing it's because this fixture in the base test function doesn't work properly in an Electron environment. Hence why I tried overriding the context fixture since Electron uses .context() instead of .newContext() but it doesn't seem to get called so I'm not sure what's going on.

For now I'm just capturing them manually

  test.beforeEach(async ({ context }, testInfo) => {
    if ((process.env.CI && testInfo.retry === 1) || !process.env.CI) {
      await context.tracing.start({ screenshots: true, snapshots: true });
    }
  });

  test.afterEach(async ({ context }, testInfo) => {
    if ((process.env.CI && testInfo.retry === 1) || !process.env.CI) {
      const testPath = testInfo.outputPath("trace.zip");
      await context.tracing.stop({ path: testPath });
    }
  });

Which works fine. (actually the tests don't seem to be retrying even with retries set to >0)

Any ideas @pavelfeldman ?

[piotrpawlik]

@pavelfeldman Probably not the best place to ask but can't find a better one. Do you have plans for fully supporting electron (exiting experimental stage) and when this could happen?

[hansiemithun]

We require screenshots, video capture on failures with the electron as it's one of the best features that playwright offers, and looking forward to being fixed in the upcoming release.

[hansiemithun]

I saw a response in my closed ticket, more votes are required to address this enhancement. How many more votes are required for such a feature which is just avoided for Electron? I believe this is sufficient and all are expecting to be released soon as it benefits the own playwright community too. Thanks in advance.

[hansiemithun]

I don't understand what is stopping this enhancement or fix which is most essential. This is not a fancy cosmetic change.

[QingTannerLi]

Looking forward to "a mode where you start once and all tests reuse the app" for electron. Currently, I referenced the code here to set electronApp as a fixture that launches in each test case. https://github.com/microsoft/playwright/blob/main/tests/electron/electronTest.ts

[steven-the-qa]

@pavelfeldman Is there a workaround we can use in the meantime for Electron tests to take advantage of the screenshot-on-failure feature? I currently have the same issue as described in Issue 12125

[iamchucky]

@boutchersj To take screenshot on failure, I think one can take advantage of the Automatic Fixture as a workaround:

import { test as base } from '@playwright/test';
import { expect } from '@playwright/test';
import { _electron as electron } from '@playwright/test';
import { ElectronApplication } from '@playwright/test';
import { Page } from '@playwright/test';

let electronApp: ElectronApplication;
let page: Page;

// Note how we mark the fixture as { auto: true }.
// This way it is always instantiated, even if the test does not use it explicitly.
export const test = base.extend<{ attachScreenshotsToReport: void }>({
  attachScreenshotsToReport: [async ({}, use, testInfo) => {
    await use();

    // After the test we can check whether the test passed or failed.
    if (testInfo.status !== testInfo.expectedStatus) {
      const screenshot = await page.screenshot();
      await testInfo.attach('screenshot', { body: screenshot, contentType: 'image/png' });
    }
  }, { auto: true }]
});

test.beforeAll(async () => {
  electronApp = await electron.launch();
  page = await electronApp.firstWindow();
});

test('example test', async () => {
  // Making test fail
  expect(true).toBe(false);
});

[deiva-s]

Since I am seeing a few linked tickets pertaining to electron and tracing, I am commenting here. Let me know if this query should go to a different place.

We are trying to automate an end to end flow which involves doing some stuff in a web app (browser) followed by doing some stuff in an electron app. We are successful in automating the flow end to end. Few challenges were encountered while trying to view the traces.

• though "playwright.config.ts" is configured for to retain traces on failure, we are not able to see the traces for the peices where we access the electron app. instead a blank browser is seen similar to https://github.com/microsoft/playwright/issues/31393

• if we try to do trace recording using the following lines, we are getting trace.zip in a separate folder and is not visible as part of the e2e flow.

  await window.context().tracing.start({ screenshots: true, snapshots: true }); await window.context().tracing.stop({ path });

• Is there a way to merge the traces of browser and electron app together? or is there any other alternative for the time being to solve this problem?

[lf94]

Is there a way playwright can expose an interface for people to give it their own [screenshot, time] pairs?... That'd give us a lot of solutions. Alternatively maybe even a video with a list of times as significant moments?