Reusable Performance Testing Environment

[swissspidy]

The existing performance testing environments in core and Gutenberg need to be further improved, and made easier for everyone else to set up similar tests by making the setup as reusable as possible. These two objectives are somewhat intertwined and require work across multiple areas.

A lot of work has been done in this regard already, but there's still a lot to do to make the setup DRY and more reusable.

Done

• [x] Track new load time metrics (TTFB, LCP) in core performance tests
• [x] Add Playwright support to @wordpress/scripts
• [x] Expand core performance tests comparison & output
• [x] Add new performance utilities to @wordpress/e2e-test-utils-playwright

In progress

• [x] Convert core performance tests to Playwright to improve stability and use the new utilities
  • https://github.com/WordPress/wordpress-develop/pull/5333
• [ ] Reduce test result variance
  • https://github.com/WordPress/performance/issues/849

To-do

• [ ] Provide new shareable utils to easily display performance test results in a comprehensive way (e.g. nice output, base comparison)
• [ ] Refactor core & Gutenberg tests to use the new utils, removing duplicated code
• [x] Provide documentation for plugin/theme developers on how to write performance tests using this toolchain
  • Use community feedback to further iterate on the setup
• [x] Provide simple GitHub Action for running benchmarks
  • Ideally using the same Playwright setup

[swissspidy]

Provide new shareable utils to easily display performance test results in a comprehensive way (e.g. nice output, base comparison)

Opened https://github.com/WordPress/gutenberg/issues/55852 for discussion.

Provide simple GitHub Action for running benchmarks

There is now https://github.com/swissspidy/wp-performance-action which uses Playwright under the hood

[swissspidy]

Provide documentation for plugin/theme developers on how to write performance tests using this toolchain

https://pascalbirchler.com/wordpress-performance-testing/