ampproject / px-toolbox-php

Apache License 2.0
8 stars 4 forks source link

Add more fixture files with Lighthouse data #22

Open ediamin opened 2 years ago

ediamin commented 2 years ago

Fixes #6

Currently we have only one single fixture file for the https:\\amp-wp.org page. The goal of this PR is to add more fixture files so we can use them with --url flag.

This PR introduces two new binaries,

  1. generate-fixture: This will spin up a WP site to test against PageSpeed Insights API. It'll create/update a fixture file psi_local-wp_amp-page.json file in tests/fixtures directory.
  2. stop-server: It cleans up/destroys everything that setup by generate-fixture. This is helpful when we work in our local.

Prerequisite:

To make the action workable in GitHub, we need to setup two secrets:

To work in local

To-dos:

ref: https://github.com/ampproject/px-toolbox-php/issues/6#issuecomment-1029498802

codecov-commenter commented 2 years ago

Codecov Report

Merging #22 (04789b4) into main (f01a864) will increase coverage by 1.84%. The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##               main      #22      +/-   ##
============================================
+ Coverage     61.25%   63.09%   +1.84%     
+ Complexity      340      338       -2     
============================================
  Files            50       50              
  Lines           831      794      -37     
============================================
- Hits            509      501       -8     
+ Misses          322      293      -29     
Flag Coverage Δ
php 63.09% <ø> (+1.84%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/PageSpeed/PageSpeedInsightsApi.php 70.00% <0.00%> (-2.73%) :arrow_down:
src/Cli/Command/Optimize.php 89.47% <0.00%> (-1.84%) :arrow_down:
src/Cli/Command/Analyze.php 0.00% <0.00%> (ø)
src/Cli/Command/PageSpeedInsights.php 0.00% <0.00%> (ø)
src/Engine/Exception/InvalidJsonFile.php 100.00% <0.00%> (ø)
src/Engine/ToolStack/ParallelToolStack.php 0.00% <0.00%> (ø)
src/Engine/Exception/FailedToProcessTimestamp.php 100.00% <0.00%> (ø)
src/Engine/StringStream.php 10.00% <0.00%> (+0.24%) :arrow_up:
src/Engine/Context.php 25.00% <0.00%> (+2.77%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f01a864...04789b4. Read the comment docs.

ediamin commented 2 years ago

Updated the generate-fixture script to check the lighthouseVersion change. Now the fixture file will only update if lighthouseVersion are different in the API data and in the existing psi_local-wp_amp-page.json in the project.

schlessera commented 2 years ago

@ediamin That looks pretty good so far. How many audits are we triggering so far with the current site setup?

ediamin commented 2 years ago

@schlessera I've added a testing plugin which will simulate bad audit scores for the following,

cumulative-layout-shift
largest-contentful-paint
first-contentful-paint
speed-index
unminified-javascript
unminified-css
uses-rel-preconnect

in addition to these, I've found that overall performance stays under 60 in multiple tests.