catchpoint / WebPageTest.bulk-tester

Google App Script for spreadsheet that uses the WPT API to bulk test URLs
MIT License
260 stars 22 forks source link

Unable to add Core Vitals metrics #27

Closed akshay-ranganath closed 4 years ago

akshay-ranganath commented 4 years ago

WebPageTest now offers the Core Web Vital metrics. These metrics are exposed as data.data.median.firstView.chromeUserTiming.* field. However, the presence of . in the name of the field is confusing the webpagetest.gs script file.

It would be helpful if the script is updated to support extracting this kind of information.

akshay-ranganath commented 4 years ago

Here is how I fixed the issue for my specific requirement. However, I think this needs a better solution to make it generic.

/**
 * Checks the status of any uncompleted tests, retrieves the results and inserts into sheet
 */

function getResults() {

    var spreadsheet = SpreadsheetApp.getActive();
    var sheet = spreadsheet.getSheetByName(TESTS_TAB);

    var range = sheet.getRange(2, 3, sheet.getLastRow() - 1, 2); // Just get URL for test, and status columns

    var urls_array = range.getValues();

    var resultsMap = getResultsMap();

    var outstandingResults = 0; // track how many tests yet to complete

    for (var i = 0; i < urls_array.length; i++) {

        var url = urls_array[i][0];
        var status = urls_array[i][1];

        if (url && status < 200) {

            // WebPageTest
            var wptAPI = url + "?f=json";

            var response = UrlFetchApp.fetch(wptAPI);
            var result = JSON.parse(response.getContentText());

            e = sheet.setActiveCell("D" + (2 + i));
            e.setValue(result.statusCode);

            if(result.statusCode < 200) {
                outstandingResults++;
            }
            else if(result.statusCode == 200) {

                for(var column in resultsMap) {
                    cell = sheet.setActiveCell(column + (2 + i));

                    if(resultsMap[column].value.indexOf('chromeUserTiming')>-1){
                        //this is from the newish chromeUserTiming section and requires looping for finding the metric.
                        var fields = resultsMap[column].value.split('.chromeUserTiming.')              
                        var metrics = eval("result." + fields[0])
                        var value = metrics['chromeUserTiming.' + fields[1]]                                    
                        if(value!=undefined){
                            cell.setValue(value)
                        }                 

                    }else{
                        var value = eval("result." + resultsMap[column].value);  // TODO: remove eval

                        // some results field may not exist in some tests e.g. SpeedIndex relies on video capture
                        if(value != undefined) {
                            cell.setValue(eval("result." + resultsMap[column].value));
                        }
                    }
                }
            }
        }
    }

    // If all tests have completed cancel the trigger
    if(outstandingResults == 0) {
        cancelTrigger()
    }
}
andydavies commented 4 years ago

It should be possible to do this without needing to add the extra code using something like this

data.median.firstView['chromeUserTiming.CumulativeLayoutShift']

But, yes the period in the variable name is causing the issue

Nooshu commented 4 years ago

This should now be fixed by using the normalizekeys=1 URL parameter (see PR29), where the above becomes:

data.median.firstView.chromeUserTimingCumulativeLayoutShift
andydavies commented 4 years ago

As @Nooshu say this is now fixed by #31