HTTPArchive / custom-metrics

Custom metrics to use with WebPageTest agents
Apache License 2.0
19 stars 22 forks source link

Add custom metrics for the WordPress Interactivity API #113

Closed adamsilverstein closed 5 months ago

adamsilverstein commented 5 months ago

Collect data about usage of the WordPress interactivity API, which recently shipped in WordPress 6.5 (see https://make.wordpress.org/core/2024/03/04/interactivity-api-dev-note/)

Adds two new datapoints to the cms/wordpress custom metric:


Test websites:

Current results:

adamsilverstein commented 5 months ago

@adamsilverstein Leaving a bit of feedback. I think some of this needs to be refined a bit more in which data we want to include. Also we should differentiate between Interactivity API "regions" and any elements that use the Interactivity API.

Thanks for the feedback, I like the idea of summarizing just to the counts by type, I'm going to build out some test pages to clarify the best way to do this.

adamsilverstein commented 5 months ago

Updates:

pmeenan commented 5 months ago

FYI, the April crawl kicks off Wednesday. Not sure if you want this metric in by then but, if so, it'll need to land today so I can update the agents before it kicks off.

tunetheweb commented 5 months ago

@adamsilverstein I updated the first comment to the correct heading needed to trigger the test ("Test websites:" rather than "Test page"). So if you push a commit now it should kick that off for your test site.

adamsilverstein commented 5 months ago

@adamsilverstein I updated the first comment to the correct heading needed to trigger the test ("Test websites:" rather than "Test page"). So if you push a commit now it should kick that off for your test site.

@tunetheweb - Ah, thanks! I probably changed that from the template because I was only linking to one website (and I didn't realize this triggered the test).

I noticed when the test action ran, the results don't include the full region counts, ie: image total_regions_by_namespace shows as empty.

Is that because the output doesn't show the full data, or something off with the code? The code did work correctly when I ran in the console. I'll try running in WPT through the UI.

tunetheweb commented 5 months ago

Is that because the output doesn't show the full data, or something off with the code? The code did work correctly when I ran in the console. I'll try running in WPT through the UI.

It's because we can't JSON.stringify an Array

The same thing happens in DevTools if you do this:

function getInteractivityAPIUsage() {
  // Look for data-wp-interactive regions.
  const interactiveRegions = document.querySelectorAll('[data-wp-interactive]');

  // Count the regions by namespace.
  const regionNamespaceCounts = []
  interactiveRegions.forEach( region => {
    // Extract the namespace from JSON or as a string
    let namespace = '';
    const regionAttribute = region.getAttribute('data-wp-interactive');
    try {
      const regionData = JSON.parse( regionAttribute );
      if ( regionData.namespace && 'string' === typeof regionData.namespace ) {
        namespace = regionData.namespace;
      }
    } catch ( e ) {
      namespace = regionAttribute;
    }
    if ( '' !== namespace ) {
      regionNamespaceCounts[ namespace ] = ( regionNamespaceCounts[ namespace ] || 0 ) + 1;
    }
  })

  console.log(regionNamespaceCounts, JSON.stringify(regionNamespaceCounts));

  return JSON.stringify({
    total_regions: interactiveRegions.length,
    total_regions_by_namespace: regionNamespaceCounts,
  });
}
getInteractivityAPIUsage();

Will suggest something.

github-actions[bot] commented 5 months ago
Custom metrics for https://almanac.httparchive.org/en/2022/ WPT test run results: http://webpagetest.httparchive.org/results.php?test=240408_EV_T
Custom metrics for https://interactivity-api.instawp.xyz/test-post/ WPT test run results: http://webpagetest.httparchive.org/results.php?test=240408_MY_V Changed custom metrics values: ```json { "_cms": { "wordpress": { "block_theme": true, "has_embed_block": false, "embed_block_count": { "total": 0, "total_by_type": [] }, "scripts": [ { "handle": "jquery-core", "src": "https://interactivity-api.instawp.xyz/wp-includes/js/jquery/jquery.min.js?ver=3.7.1", "in_footer": false, "async": false, "defer": false, "intended_strategy": null, "after_script_size": null, "before_script_size": null, "extra_script_size": null, "translations_script_size": null }, { "handle": "jquery-migrate", "src": "https://interactivity-api.instawp.xyz/wp-includes/js/jquery/jquery-migrate.min.js?ver=3.4.1", "in_footer": false, "async": false, "defer": false, "intended_strategy": null, "after_script_size": null, "before_script_size": null, "extra_script_size": null, "translations_script_size": null }, { "handle": "ideabox-toc-script", "src": "https://interactivity-api.instawp.xyz/wp-content/plugins/table-of-contents/assets/js/frontend.js?ver=1.0.2", "in_footer": false, "async": false, "defer": false, "intended_strategy": null, "after_script_size": null, "before_script_size": null, "extra_script_size": null, "translations_script_size": null } ], "content_type": { "template": "unknown", "post_type": "", "taxonomy": "" }, "uses_interactivity_api": true, "interactivity_api_usage": { "total_regions": 12, "total_regions_by_namespace": { "core/navigation": 4, "myPlugin": 1, "core/search": 1, "core/image": 2, "core/query": 2, "core/router": 2 } } } } } ```
Custom metrics for https://github.com/HTTPArchive/custom-metrics/assets/2676022/65913bd4-eb52-4154-9438-b462f702e80c WPT test run results: http://webpagetest.httparchive.org/results.php?test=240408_QS_W Changed custom metrics values: ```json { "_cms": { "wordpress": { "block_theme": false, "has_embed_block": false, "embed_block_count": { "total": 0, "total_by_type": [] }, "scripts": [], "content_type": { "template": "unknown", "post_type": "", "taxonomy": "" }, "uses_interactivity_api": false, "interactivity_api_usage": { "total_regions": 0, "total_regions_by_namespace": [] } } } } ```
adamsilverstein commented 5 months ago

It's because we can't JSON.stringify an Array

@tunetheweb Ah, right. This has caused me problems before (and yet, I tried again).

return JSON.stringify(

I'm not clear on this bit: we don't stringify other returns, for example the embed block data returned from getWordPressEmbedBlockCounts above... why is JSON.stringify needed here? (the data structure seems very similar to the embed data)

tunetheweb commented 5 months ago

Ah sorry you don't need to JSON.stringify each function separately. I was testing with just the getInteractivityAPIUsage function call, and not the whole thing.

You SHOULD however JSON.stringify at the end.

return JSON.stringify({
  wordpress
};

JavaScript objects are NOT the same as JSON. They look very similar, but not quite the same. A JavaScript object is a structured object which can include references. JSON is just a string in a particular format. Not every JavaScript object can be cleanly converted to a JSON string.

Which is what we saw earlier, and what we can repeat here with this smaller bit of code:

const myArray = [];
myArray['subArray1'] = [1,2,3];
myArray['subArray2'] = [4,5,6];
console.log(myArray);
console.log(JSON.stringify(myArray));
image

The first line prints named arrays, the second just an empty array since you can't convert named arrays to JSON (they are basically parameters on the array variable, rather than values in the array).

The length of that array is 0 btw:

image

Because you are NOT pushing values in that array, you are setting attributes on that array. And when you stringify an array, it only looks at the values, not the attributes.

This however does work:

const myArray = [];
myArray[0] = [1,2,3];
myArray[1] = [4,5,6];
console.log(myArray);
console.log(JSON.stringify(myArray));
image

Anyway, HTTP Archive stores these as Strings. In most cases it will convert these automatically which is why you don't NEED it here. However I advise to use JSON.stringify() at the end. Why? Because then when you test this in the console you'll also see when it's missing stuff you expected to in there rather than be confused why WebPageTest's magic string conversion (which isn't magic at all) did something different than you expected.