search

vaadin / web-components

A set of high-quality standards based web components for enterprise web applications. Part of Vaadin 20+
https://vaadin.com/docs/latest/components
470 stars 83 forks source link

Upgrade Playwright version #7716

Open sissbruecker opened 2 months ago

sissbruecker commented 2 months ago

We're currently using "@web/test-runner-playwright": "^0.11.0", which pulls in playwright "^1.22.2". That version was released over two years ago and also uses browser versions (we use PW for Firefox and Webkit tests) that are over two years old at this point.

We should upgrade Playwright to the latest version and use updated browser versions for testing. From testing this there are a number of test failures that need to be resolved. These seem to mostly revolve around focus handling and native key presses:

Firefox failures ❌ validation > basic > should validate on focusout ❌ validation > basic > should validate on focusout ❌ vaadin-checkbox-group > focused attribute > should remove focused attribute on Shift+Tab ❌ validation > basic > should validate on focusout ❌ validation > basic > should validate on focusout ❌ vaadin-checkbox-group > focused attribute > should remove focused attribute on Shift+Tab ❌ interactions > virtual keyboard > should re-enable virtual keyboard on blur ❌ interactions > virtual keyboard > should re-enable virtual keyboard on blur ❌ keyboard > navigating after overlay opened > should tab to the next focusable ❌ keyboard > navigating after overlay opened > should tab to the next focusable ❌ reorder elements > focus > should tab through the elements in order ❌ vaadin-cookie-consent > custom properties > should close on Space key after change ❌ dropdown > virtual keyboard > should re-enable virtual keyboard on blur ❌ dropdown > virtual keyboard > should re-enable virtual keyboard on blur ❌ user-tags > inside a scrollable container > adding users when the field is partially visible and focused > should open the overlay when the field is fully visible and hide otherwise ❌ keyboard navigation > interaction mode > should tab through the elements in order ❌ keyboard navigation > interaction mode > should tab through the elements in order ❌ edit column > select column > should focus previous cell available for editing in edit mode on Shift Tab ❌ edit column > select column > should focus previous cell available for editing in edit mode on Shift Tab ❌ vaadin-icon - icon fonts > sizing > should subtract vertical padding from height ❌ vaadin-icon - icon fonts > sizing > should subtract vertical padding from height ❌ readonly > basic > should close the dropdown on Tab when readonly ❌ keyboard navigation > Tab > should leave focusable only the checked radio button ❌ keyboard navigation > Tab > should leave focusable only the checked radio button ❌ radio-group > focused state > should remove focused attribute on Shift+Tab ❌ validation > basic > should validate on focusout ❌ radio-group > focused state > should remove focused attribute on Shift+Tab ❌ validation > basic > should validate on focusout ❌ validation > basic > should validate once on close with Tab ❌ validation > basic > should validate once on close with Tab ❌ text-area > multi-line > should not change height ❌ text-area > multi-line > should change height automatically on width change ❌ text-area > multi-line > should grow height with unwrapped text ❌ text-area > multi-line > should stay between min and max height ❌ text-area > multi-line > should maintain scroll top ❌ text-area > multi-line > should decrease height automatically ❌ text-area > multi-line > should change height automatically on width change ❌ text-area > multi-line > --_text-area-vertical-scroll-position CSS variable > should update value on scroll ❌ text-area > multi-line > --_text-area-vertical-scroll-position CSS variable > should update value on wheel ❌ text-area > multi-line > --_text-area-vertical-scroll-position CSS variable > should scroll on wheel ❌ text-area > multi-line > --_text-area-vertical-scroll-position CSS variable > should cancel wheel event ❌ virtual-list > with items > should have a last visible index ❌ virtual-list > with items > should have a last visible index
Webkit failures ❌ validation > basic > should validate on focusout ❌ validation > basic > should validate on focusout ❌ validation > basic > should validate on focusout ❌ validation > basic > should validate on focusout ❌ vaadin-checkbox-group > focused attribute > should remove focused attribute on Shift+Tab ❌ vaadin-checkbox-group > focused attribute > should remove focused attribute on Shift+Tab ❌ interactions > virtual keyboard > should re-enable virtual keyboard on blur ❌ interactions > virtual keyboard > should re-enable virtual keyboard on blur ❌ keyboard > navigating after overlay opened > should tab to the next focusable ❌ keyboard > navigating after overlay opened > should tab to the next focusable ❌ a11y > focus restoration > should move focus to the prev element outside the menu on Shift+Tab pressed inside ❌ a11y > focus restoration > should move focus to the next element outside the menu on Tab pressed inside ❌ a11y > focus restoration > should move focus to the prev element outside the menu on Shift+Tab pressed inside ❌ a11y > focus restoration > should move focus to the next element outside the menu on Tab pressed inside ❌ vaadin-cookie-consent > custom properties > should close on Space key after change ❌ dropdown > virtual keyboard > should re-enable virtual keyboard on blur ❌ dropdown > virtual keyboard > should re-enable virtual keyboard on blur ❌ InputFieldMixin + Polymer > attributes > autocorrect > should set autocorrect property on the host element ❌ InputFieldMixin + Polymer > attributes > autocorrect > should propagate autocorrect attribute to the input ❌ empty state > should tab to the next focusable element in empty state content ❌ empty state > should tab to the next focusable element in empty state content ❌ focus > should remove focus-ring when a child node is focused ❌ readonly > basic > should close the dropdown on Tab when readonly ❌ Tests were interrupted because the page navigated to about:blank. This can happen when clicking a link, submitting a form or interacting with window.location. ❌ keyboard navigation > Tab > should leave focusable only the first button by default ❌ keyboard navigation > Tab > should leave focusable only the checked radio button ❌ keyboard navigation > Tab > should leave focusable only the first button by default ❌ keyboard navigation > Tab > should leave focusable only the checked radio button ❌ validation > basic > should validate on focusout ❌ validation > basic > should validate on focusout ❌ radio-group > focused state > should remove focused attribute on Shift+Tab ❌ accessibility > keyboard navigation > should move focus to next element after esc followed by tab are pressed ❌ accessibility > keyboard navigation > should restore default Tab behavior after multiple Esc and then Tab ❌ radio-group > focused state > should remove focused attribute on Shift+Tab ❌ validation > basic > should validate once on close with Tab ❌ accessibility > focus > should delegate focus to the native button element in shadow root ❌ accessibility > focus > should set focused and focus-ring attributes when keyboard focused ❌ validation > basic > should validate once on close with Tab ❌ element > focus > should remove focus-ring when a button is focused ❌ element > focus > should not set focus-ring on Shift Tab to button ❌ keyboard navigation > file > should focus on the start button ❌ keyboard navigation > file > should focus on the retry button ❌ keyboard navigation > file > should focus on the clear button ❌ element > focus > should remove focus-ring when a button is focused ❌ element > focus > should not set focus-ring on Shift Tab to button ❌ keyboard navigation > file > should focus on the start button ❌ keyboard navigation > file > should focus on the retry button ❌ keyboard navigation > file > should focus on the clear button ❌ Browser tests did not finish within 120000ms. You can increase this timeout with the testsFinishTimeout option. Check the browser logs or open the browser in debug mode for more information.

I've reported a related regression in the Playwright repo, once that is resolved we can check again if that improves things.

yuriy-fix commented 2 months ago

When upgrading to the latest version, let's consider version bump automation for example, via dependabot.

web-padawan commented 1 month ago

The issue linked above was merged into this one (which is still not fixed at the time of writing):