search

ampproject / px-toolbox-php

Apache License 2.0
8 stars 4 forks source link

Add more fixture files with Lighthouse data #6

Open schlessera opened 2 years ago

schlessera commented 2 years ago

Feature Description

We currently have 1 single fixture file for the https:\\amp-wp.org page.

We should extend this to have multiple pages across multiple real domains as fixture files, so that we can use the --url flag to choose one of multiple fixture files. These should include official pages from the AMP and/or WordPress project.

Ideally, we would have enough fixture coverage to have most or all of Lighthouse's audits be triggered for producing data.

To ensure we can easily maintain this, the fixture files should be generated via a script that we can trigger at a later point to update the fixture files.

schlessera commented 2 years ago

Based on the discussion with @westonruter & @ediamin so far, this is the current plan:

  1. Build a script to easily pull up a WordPress server locally that can be made accessible to the public. A possible implementation might use WP-CLI to set up a server, use the wp server command to serve it locally, and then use ngrok to make that server publically accessible.
  2. Build a script to audit a URL via the PageSpeed Insights API and store the JSON result under the tests/fixtures folder, using the URL and the versions of both PageSpeed Insights API and the embedded Lighthouse audit.
  3. Use the AMP plugin reference site implementation to build a "worst possible site" that can trigger all warnings that Lighthouse has to offer. This can also include settings or hacks in ngrok to slow down the network stack.
  4. Build a GitHub Actions workflow that pulls up a server via 1., runs the audit from 2., checks whether there were changes in the tests/fixtures folder, and if so, create a new pull request to add that new fixture file to the repo. This can be based on a similar workflow in the AMP toolbox repo: https://github.com/ampproject/amp-toolbox-php/blob/main/.github/workflows/sync-spec-test-suite.yml

This will allow us to have consistent fixture files to test against, and that keep up-to-date automatically with the latest PSI/Lighthouse changes.

ediamin commented 2 years ago

@schlessera Need some help for the point 3. I'm currently using the wp post create to generate a post from a html file with some content. Could you please suggest me what content should I use to generate fixture file? What content should I use to build the "worst possible site"?

ediamin commented 2 years ago

Update: Working on the GitHub workflow. It is almost finished. Tested it in a fork and it is working fine so far. After this I'll build the WordPress content against which we will run the PageSpeed Insight tests.