WordPress / performance

Performance plugin from the WordPress Performance Group, which is a collection of standalone performance modules.
https://wordpress.org/plugins/performance-lab/
GNU General Public License v2.0
365 stars 100 forks source link

Snapshot testing for Optimization Detective #1418

Open westonruter opened 3 months ago

westonruter commented 3 months ago

With https://github.com/WordPress/performance/pull/1405, the Optimization Detective tests were refactored to be easier to maintain by putting the test cases into separate files. However, when changes occur to the actual HTML output they remain tedious to update. We should consider using phpunit-snapshot-assertions to make this easier. There is even an assertMatchesHtmlSnapshot() assertion we can use instead of assertEquals which does normalization which would reduce the need to use remove_initial_tabs. (Nevertheless, the HtmlDriver is using PHP DOM which has limitations.)

Relevant code that this would touch (in addition to the test-cases subdirectories):

  1. https://github.com/WordPress/performance/blob/e34711dbf286a16523b3c3d35462e0e79ea4c6f9/plugins/embed-optimizer/tests/test-optimization-detective.php#L40-L80
  2. https://github.com/WordPress/performance/blob/e34711dbf286a16523b3c3d35462e0e79ea4c6f9/plugins/image-prioritizer/tests/test-helper.php#L32-L65
  3. https://github.com/WordPress/performance/blob/e34711dbf286a16523b3c3d35462e0e79ea4c6f9/plugins/optimization-detective/tests/test-optimization.php#L188-L245
  4. https://github.com/WordPress/performance/blob/e34711dbf286a16523b3c3d35462e0e79ea4c6f9/plugins/auto-sizes/tests/test-optimization-detective.php#L43-L122
swissspidy commented 3 months ago

There is even an assertMatchesHtmlSnapshot() assertion we can use instead of assertEquals which does normalization which would reduce the need to use remove_initial_tabs. (Nevertheless, the HtmlDriver is using PHP DOM which has limitations.)

As a start we could also think about writing our own custom assertions first, or write a custom driver for that package.

westonruter commented 3 months ago

It's interesting that assertMatchesHtmlSnapshot() ends up using assertEquals on the serialized HTML, rather than comparing DOMDocument instances.

I recall that @dmsnell also maybe has implemented such an HTML-equivalency algorithm using HTML (Tag?) Processor?

dmsnell commented 3 months ago

@westonruter we have primitive forms with the Tag Processor, but I think that the HTML Processor will, for the most part, support all HTML in 6.7, making it possible to propose something in Core for a new assertion.

PHP 8.4 will have a spec-compliant HTML5 parser based on lexbor which will also do the trick and support everything, but in most cases I believe we will have an effective tool with the HTML API, but it doesn't exist yet in a proposal.