microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
67.21k stars 3.7k forks source link

Current limitations blocking Playwright's WebDriver BiDi adoption #32577

Open yury-s opened 2 months ago

yury-s commented 2 months ago

We've spent a couple weeks adding support for Bidi to Playwright and running Playwright tests with the new protocol. The good news is there has been a large progress in Bidi support in Firefox and Chromium, many Playwright tests are now passing. However, there is still a long list of issues that are blocking Playwright from adopting Bidi as the browser automation protocol. At this point the number of bugs we identified is too large to file individual reports for each of them, instead we'll use this issue to describe some of the major problems that we've discovered.

🎉 Current stats are the following:

Spec

These are just some of the issue that we discovered while looking at the first cut. There are many other issues that are currently blocked by the ones above.

Implementation

whimboo commented 2 months ago

To mention it here as well... Thanks a lot for all the details @yury-s! This is pretty helpful. As stated on the BiDi issue already I've created a Google document that contains all the above details, for collaborative editing. If you could find the time to at least roughly prioritize those features that would be great. You have a better idea what's most important, but I'll also dig into and see what could be the main issues with Firefox specifically to get more tests passing. Thanks!

yury-s commented 2 months ago

@whimboo I've assigned priorities to some items in the document. Since these are the first major issues we encountered during testing, and many are blockers for further progress, most are marked as P1 or P0. For clarity, I've added definition for the priority values used. Those are priorities from Playwright's perspective. For Browser implementation issues, I highlighted those that result in massive failures (and should be prioritized higher). Beyond that I'd defer prioritization of individual feature bugs to the browser maintainers.

whimboo commented 1 month ago

@yury-s some of the listed entries for Firefox are fixed in the meantime. We probably should get the list above updated (items checked) or fully rely on the Google document. It's hard to sync items on both sides - and here I don't have permissions to edit your comment for that purpose.

yury-s commented 1 month ago

@whimboo thanks for bringing this up, I went ahead and marked resolved items, they should know be in sync with the Google doc.