microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.9k stars 3.67k forks source link

[BUG] background-attachment:fixed prevents scrolling viewport on webkit with isMobile: true #23573

Closed jamesbirtles closed 1 month ago

jamesbirtles commented 1 year ago

System info

Source code

Link to the GitHub repository with the repro

https://github.com/jamesbirtles/playwright-webkit-mobile-scroll-to-click

Steps

Expected

Tests should pass on all device configurations

Actual

Test fails on "Mobile Safari" device.


It appears the page is not being scrolled (whilst trying to bring the button at the bottom of the page into view in order to click it) on the webkit browser when isMobile is true and background-attachment:fixed is applied (at least to html or body element). You can see this in my example repo. A background image needs to be set to reproduce this, but it doesn't matter if the url actually resolves to anything.

You can see in this github action run - https://github.com/jamesbirtles/playwright-webkit-mobile-scroll-to-click/actions/runs/5201582813/jobs/9381960009 - that this only appears to happen when running on linux. This does not occur locally on my M2 mac, nor on the macos-latest github runner. I can reproduce locally by running a linux docker container.

The issue only appears to occur on mobile webkit, it does not appear on mobile chromium or desktop variants of either.

edumserrano commented 1 year ago

Any updates on this? Any workaround, other than not using background-attachment:fixed or setting isMobile to false on the playwright config for the project? I'm having this issue on playwright 1.35.1.

I thought I was going crazy when I came across this issue. The tests were running fine until we did a simple CSS change to add the background-attachment:fixed and suddenly lots of tests started to fail on the projects using webkit with isMobile set to true. The tests would fail at the timeout with errors like:

waiting for getByRole('button', { name: 'Back' })
  locator resolved to <button size="xl" type="button" _ngcontent-bro-c3="" ng-…>…</button>
attempting hover action
  waiting for element to be visible and stable
  element is visible and stable
  scrolling into view if needed
  done scrolling
  element is outside of the viewport
retrying hover action, attempt #1
  waiting for element to be visible and stable
  element is visible and stable
  scrolling into view if needed
  done scrolling
  element is outside of the viewport
retrying hover action, attempt #2
  waiting 20ms
  waiting for element to be visible and stable
  element is visible and stable
  scrolling into view if needed
  done scrolling
  element is outside of the viewport
retrying hover action, attempt #3
  waiting 100ms
  waiting for element to be visible and stable
  element is visible and stable
  scrolling into view if needed
  done scrolling
  element is outside of the viewport
Call log:
  - expect.toHaveScreenshot(two-sign-in-options.png) with timeout 40000ms
  -   generating new stable screenshot expectation
  - taking page screenshot
  -   disabled all CSS animations
  - waiting 100ms before taking screenshot
  - taking page screenshot
  -   disabled all CSS animations
  - 46695 pixels (ratio 0.20 of all image pixels) are different.
  - waiting 250ms before taking screenshot
  - taking page screenshot
  -   disabled all CSS animations
  - 46308 pixels (ratio 0.20 of all image pixels) are different.
  - waiting 500ms before taking screenshot
  - taking page screenshot
  -   disabled all CSS animations
  - 46308 pixels (ratio 0.20 of all image pixels) are different.
  - waiting 1000ms before taking screenshot
  - taking page screenshot
  -   disabled all CSS animations
  - 46308 pixels (ratio 0.20 of all image pixels) are different.
  - waiting 1000ms before taking screenshot
  - taking page screenshot
aslushnikov commented 1 year ago

@JoelEinbinder any ideas?

JoelEinbinder commented 1 year ago

@JoelEinbinder any ideas?

No thank you. gl

fabbaumgartner commented 10 months ago

Are there updates on this issue?

TomTom-Labs commented 5 months ago

I am running in the same issue. @edumserrano Did you find a workaround, other than disable all tests on "Safari Mobile"?

edumserrano commented 5 months ago

We did not @TomTom-Labs. Here's the comment we left on the playwright.config:


// playwright.config.ts

projects: [
  /*
   * Warning:
   *
   * We've had some issues testing with webkit in resolutions lower than 1920x1080, mainly on ipad mini. The two issues were either:
   * 1) problems with font kerning. See https://github.com/microsoft/playwright/issues/20203 and https://github.com/microsoft/playwright/issues/23789.
   * 2) problems with specific CSS which we have in our app and its interaction with webkit. See https://github.com/microsoft/playwright/issues/23573.
   *
   * To overcome the above issues we considered:
   * A) disabling `isMobile` in the iphone resolution
   * B) in all resolutions if the browser was webkit, before taking screenshots, we would do a `await page.waitForTimeout(<timeoutValue>>);`. However,
   * the `timeoutValue` that would work for one machine would not necessarily work for another one. We started with 1.5s and when more team members tested
   * we needed to increase above that.
   *
   * Since we didn't like the timeout approach, we decided to NOT test with webkit (safari) below 1920 width resolutions. This can be revisited once the
   * issues are fixed or we find a better alternative to deal with the font kerning problem.
   *
   */
   <projects configuration>
]
TomTom-Labs commented 5 months ago

As an update - I found my issue (took me some time)

Long story short: I’m using tailwind css and replaced overflow-visible by overflow-auto

The problem at the end turned out that the scrollbar was missing. So, the failing test of playwright was actually correct and pointed to a real issue (not able for the user to scroll down). After enabling the scrollbar again, the user and playwright can scroll to the button and press it.

Why this was not a problem on the playwright tests on the other mobile browser is still a secret to me.