[BUG] Black Screen in video and trace on mobile

israelKusayev commented 2 years ago

Context:

Playwright Version: 1.18.1
Operating System: bitbucket pipelines + running with playwright official image mcr.microsoft.com/playwright:v1.18.1-focal
Browser: [e.g. All, Chromium, Firefox, WebKit]
Extra: project config
```
{
name: 'Mobile Safari',
use: {
    ...devices['iPhone 13 Pro']
}
}
```
When I run the tests inside bitbucket pipelines after a few seconds the video is turning to a black screen (sometimes it's totally black sometimes you can see parts of the screen) It doesn't happen when on Desktop Chrome

https://user-images.githubusercontent.com/42112796/152193451-5f12d861-0f98-4349-aae5-97cc3a978ff7.mov

israelKusayev commented 2 years ago

It's reproducible it happens every time for every mobile test

yury-s commented 2 years ago

Does it happen only with WebKit or other browsers too? Can you also share full config?

israelKusayev commented 2 years ago

As far as I tested it seems like it happens only on WebKit I'll report if I see this happens on another browser

Here is the config file:

import { PlaywrightTestConfig, devices } from '@playwright/test';

const config: PlaywrightTestConfig = {
    testDir: './src/tests',
    globalSetup: './src/tests/setup/globalSetup.ts',
    timeout: 240 * 1000,
    expect: {
        timeout: 5000
    },
    forbidOnly: !!process.env.CI,
    retries: process.env.CI ? 1 : 0,

    reporter: [['html'], ['list'], ['./src/reporters/slack-PD-reporter.ts']],

    use: {
        launchOptions: {
            // slowMo: 20
        },
        actionTimeout: 0,

        trace: 'on-first-retry',
        video: 'on-first-retry',
        screenshot: 'only-on-failure'
    },

    projects: [
        {
            name: 'chromium',
            use: {
                ...devices['Desktop Chrome']
            }
        },
        {
            name: 'Mobile Safari',
            use: {
                ...devices['iPhone 13 Pro']
            }
        }
    ]
};
export default config;

yury-s commented 2 years ago

Would you be able to share a minimal repro? I tried with a few sites locally, but to no avail.

israelKusayev commented 2 years ago

It’s not happening locally (on my Mac)

Only on bitbucket pipelines Also I’m running on the official playwright docket image so it can be either something with bitbucket pipelines operation system or the docket image (or both)

also it starts the black screen after few seconds (let’s say 10, so make sure you check it on long e2e tests (mines are average of 2 minutes for each test))

also there is a chance that it maybe related to the parallel run (I can check if it happens when I run only one test)

israelKusayev commented 2 years ago

I have a lot of problems when running iOS on bitbucket pipelines For example #11899

(Also a lot of the tests which usually work fine on chromium starts to be flaky

israelKusayev commented 2 years ago

Sorry accidentally closed the issue

israelKusayev commented 2 years ago

Also this issue that we talked about on the slack channel is happening there (I haven’t open a bug about is yet) https://playwright.slack.com/archives/C01AG7QGW2F/p1643805613719309?thread_ts=1643801931.811619&channel=C01AG7QGW2F&message_ts=1643805613.719309

dz1dz1on commented 2 years ago

Hi @yury-s, I have the same issue like @israelKusayev described above.

Context:

playwright 1.19.2 and 1.22.0 (checked on both versions)
checked on iPhone 6 (device emulation) + WebKit
problem only appears on CI in headless mode. In my case Github Actions + Google Cloud Run.

Tests starts properly but after some time (about 20 s) it goes partially black on scrolling down to the element on page(you can see the result on screenshots). Later it turns totally black for some time and it backs to partially black. On the last picture is a trace view from the run with the timeline.

Funny thing is that when I start the same test on CI in headed mode everything works smoothly - video without black screen.

PICS:

mspina-cit commented 2 years ago

I'm facing the same issue here. Usually the page gets black when scrolling down to take a locator screenshot, for exemple, when it's below the current/initial viewport. But sometimes it works, so it is intermittent. Looks like it happens more often when running multiple tests with more workers (I usually use 4 or 5 workers).

I'm using:

Webkit emulating iPhone 11
Playwright 1.19.2
Only happens when running inside docker (locally or Jenkins), using image: mcr.microsoft.com/playwright:v1.19.0-focal (when running locally in my Mac outside the docker container it goes fine)

loganromero commented 2 years ago

I'm also running into this issue.

finsterwalder commented 2 years ago

Same problem here. Running playwright locally on Linux or in a docker container. It only happens on some tests. Maybe related to opening a Modal on the web site under test?

Qibaoooo commented 2 years ago

Similar issue here. Some more context:

My test steps are: opening a home page, clicking on a Login button, another new page will be opened for user to login.

I am testing mainly on iOS Safari/webkit.

So, what I have observed is when ever I run the test inside docker (mcr.microsoft.com/playwright:v1.25.0-jammy), the new page opened will always be rendered with issue (sometimes the page is enlarged, sometimes it is minimized), and I am not able to locate any elements on the new page.

But, the test always runs fine on my mac machine.

Hope these context helps with your debugging. Thanks.

TJHdev commented 2 years ago

I'm having a very similar issue when running mobile safari tests in the CI (linux distro). I can't recreate the issue locally when running on macOs 11.4. The issue seems to crop up when toasts or popovers are involved. I can share a couple of traces with videos included but prefer not to post it publicly on here.

Playwright gets into a loop doing the following, but the video shows that the app seems to have a gone out of the bounds recording box. (very similar to what dz1dz1on has posted above)

waiting for selector "div[role="menu"] >> div[role="button"]:has-text("Delete")"
selector resolved to visible <div tabindex="0" role="button" class=" css-17j40lu" …>…</div>
attempting click action
waiting for element to be visible, enabled and stable
element is visible, enabled and stable
scrolling into view if needed
done scrolling
element is outside of the viewport
retrying click action, attempt #1
waiting for element to be visible, enabled and stable
element is visible, enabled and stable
scrolling into view if needed
done scrolling
element is outside of the viewport
retrying click action, attempt #2
waiting 20ms
waiting for element to be visible, enabled and stable
element is visible, enabled and stable
scrolling into view if needed
done scrolling

Playwright 1.27.1

gabriel-finch-kr commented 1 year ago

We also came across this problem recently with v1.32.1 using the provided docker image. After some experimentation we found that by setting isMobile: false for a given webkit device we would no longer experience the issue. To put it another way this project in config would reproduce the blank screen in HTML report/trace:

projects: [
    {
      name: 'Mobile Safari',
      use: {
        userAgent: devices['iPhone 13 Pro'].userAgent,
        viewport: devices['iPhone 13 Pro'].viewport,
        deviceScaleFactor: devices['iPhone 13 Pro'].deviceScaleFactor,
        isMobile: devices['iPhone 13 Pro'].isMobile,
        hasTouch: devices['iPhone 13 Pro'].hasTouch,
        defaultBrowserType: devices['iPhone 13 Pro'].defaultBrowserType,
      },
    },
  ]

And this modification would resolve the issue:

projects: [
    {
      name: 'Mobile Safari',
      use: {
        userAgent: devices['iPhone 13 Pro'].userAgent,
        viewport: devices['iPhone 13 Pro'].viewport,
        deviceScaleFactor: devices['iPhone 13 Pro'].deviceScaleFactor,
        isMobile: false,
        hasTouch: devices['iPhone 13 Pro'].hasTouch,
        defaultBrowserType: devices['iPhone 13 Pro'].defaultBrowserType,
      },
    },
  ]

Hopefully this info is of some use in helping to track down the issue and as a temporary workaround for folks encountering the issue.

neurolabs commented 1 year ago

Playing around with another predefined device descriptor, I found another workaround:

    name: 'Mobile_WebKit',
    use: {
      ...devices['iPhone 12'],
      // same as screen:
      viewport: {
        width: 390,
        height: 844,
      },
    },
  },

The workaround from @gabriel-finch-kr (setting isMobile to false) did not work for me with the iPhone 12 device descriptor.

luixo commented 1 year ago

I get the same problem on Macbook M1, running following:

docker run --rm --network host --platform=linux/aarch64 -v $(pwd):/work/ -w /work/ -it mcr.microsoft.com/playwright:v1.31.0-focal /bin/bash
npm install
npx playwright test --update-snapshots

My playwright.config.ts (basically the one from the docs) with only one project left (the problem one):

import { defineConfig, devices } from "@playwright/test";

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({
  testDir: "./tests",
  /* Maximum time one test can run for. */
  timeout: 30 * 1000,
  expect: {
    /**
     * Maximum time expect() should wait for the condition to be met.
     * For example in `await expect(locator).toHaveText();`
     */
    timeout: 5000,
  },
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: "html",
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Maximum time each action such as `click()` can take. Defaults to 0 (no limit). */
    actionTimeout: 0,
    /* Base URL to use in actions like `await page.goto('/')`. */
    baseURL: "http://localhost:3000/",

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: "on-first-retry",
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: "Mobile Safari",
      use: { ...devices["iPhone 12"] },
    },
  ],
});

My test:

import { test, expect } from "@playwright/test";

test.describe("Main page", () => {
  test("looks as expected", async ({ page }) => {
    await page.goto('/');
    await page.waitForLoadState("networkidle");
    await expect(page).toHaveScreenshot({ fullPage: true });
  });
});

As a result I get completely transaparent page as a screenshot. Adding isMobile: false to the project helps, but it shouldn't be like that, should it?

tomer555 commented 1 year ago

+1

Theiaz commented 1 year ago

Any updates on this?

ericvoshall commented 1 year ago

Same issue.

josephmarshall-e2x commented 8 months ago

I have the same issue on Github Actions, Playwright 1.42.0

It always seems to be when the test scrolls down the page, as others have said.

It makes it very difficult to debug Safari tests, which are already prone to a lot of issues when run in CI.

microsoft / playwright

[BUG] Black Screen in video and trace on mobile #11812