search

sitespeedio / sitespeed.io

sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.
https://www.sitespeed.io/
MIT License
4.67k stars 591 forks source link

Couldn't generate Html Report #4078

Open Pearlking311 opened 5 months ago

Pearlking311 commented 5 months ago

Have you read the documentation?

URL

https://www.demoblaze.com

What are you trying to accomplish

I have used measure.start and measure.stop in each page. When the script fails before url change, html metrics are not generated.

NOTE: I have manually failed step 3 by inputting incorrect Xpath.

What browser did you use?

Chrome

How to reproduce

Run the below mentioned script for 2 Iteration and verify Report is generated when the script fails

module.exports = async function(context, commands) 
{
    //Step 01: Landing Page
    await commands.measure.start('SC01_T01_LandingPageLaptops');
    await commands.navigate("https://www.demoblaze.com");
    await commands.measure.stop();
    context.log.info('Step 01: Success');

    //Step 02: Select Category
    await commands.measure.start('SC01_T02_SelectCategoryLaptops');
    try
    {
        await commands.click.byXpathAndWait('//a[text()="Laptops"]');
    }
    catch(e)
    {
        context.log.error('Step 02: Could not Select Category');
    }
    await commands.measure.stop();
    context.log.info('Step 02: Category selected successfully');

    //Step 03: Select Product
    await commands.measure.start('SC01_T03_SelectProductLaptops');
    try
    {
        await commands.click.byXpathAndWait('//*[@id="tbodyid"]/div[1]/div/div/h4/b');
    }
    catch(e)
    {
        context.log.error('Step 03: Could not Select Product');
        await commands.measure.stop();
        return commands.markAsFailure('Failed during Product selection');
    }
    await commands.measure.stop();
    context.log.info('Step 03: Product page displayed');
    await commands.cache.clear();
}

Log output

[2024-02-07 22:51:38] INFO: Versions OS: win32 10.0.22621 nodejs: v21.2.0 sitespeed.io: 30.6.0 browsertime: 19.1.0 coach: 8.0.2
[2024-02-07 22:51:39] INFO: Running tests using Chrome - 2 iteration(s)
[2024-02-07 22:51:40] INFO: Start to measure SC01_T01_LandingPageLaptops
[2024-02-07 22:51:40] INFO: Navigating to url https://www.demoblaze.com iteration 1
[2024-02-07 22:51:52] INFO: Take after page complete check screenshot
[2024-02-07 22:51:54] INFO: Take cumulative layout shift screenshot
[2024-02-07 22:51:57] INFO: Take largest contentful paint screenshot
[2024-02-07 22:52:00] INFO: Step 01: Success
[2024-02-07 22:52:00] INFO: Start to measure SC01_T02_SelectCategoryLaptops
[2024-02-07 22:52:05] INFO: Take after page complete check screenshot
[2024-02-07 22:52:07] INFO: Take cumulative layout shift screenshot
[2024-02-07 22:52:10] INFO: Take largest contentful paint screenshot
[2024-02-07 22:52:12] INFO: Step 02: Category selected successfully
[2024-02-07 22:52:12] INFO: Start to measure SC01_T03_SelectProductLaptops
[2024-02-07 22:52:12] ERROR: Couldn't execute script named CUSTOM error:JavascriptError: javascript error: Cannot read properties of null (reading 'click')
  (Session info: chrome=121.0.6167.140)
[2024-02-07 22:52:12] ERROR: Could not find element by xpath //*[@id="tbodyid"]/div[1]/div/div/h4/b
[2024-02-07 22:52:12] ERROR: Step 03: Could not Select Product
[2024-02-07 22:52:12] INFO: https://www.demoblaze.com/# has been tested before within the same run, it will get an extra query parameter named browsertime_run. Make sure to use alias to keep track of the URLs
[2024-02-07 22:52:12] INFO: Take after page complete check screenshot
[2024-02-07 22:52:15] INFO: Take cumulative layout shift screenshot
[2024-02-07 22:52:17] INFO: Take largest contentful paint screenshot
[2024-02-07 22:52:21] INFO: https://www.demoblaze.com/ TTFB: 1.39s DOMContentLoaded: 3.78s firstPaint: 2.00s FCP: 2.00s LCP: 2.63s Load: 5.79s CLS:0.0103
[2024-02-07 22:52:21] INFO: https://www.demoblaze.com/# TTFB: 1.39s DOMContentLoaded: 3.78s firstPaint: 2.00s FCP: 2.00s LCP: 2.63s Load: 5.79s CLS:0.0103
[2024-02-07 22:52:21] INFO: https://www.demoblaze.com/#?browsertime_run=2 TTFB: 1.39s DOMContentLoaded: 3.78s firstPaint: 2.00s FCP: 2.00s LCP: 2.63s Load: 5.79s CLS:0.0103
[2024-02-07 22:52:22] INFO: Start to measure SC01_T01_LandingPageLaptops
[2024-02-07 22:52:22] INFO: Navigating to url https://www.demoblaze.com iteration 2
[2024-02-07 22:52:37] INFO: Take after page complete check screenshot
[2024-02-07 22:52:39] INFO: Take cumulative layout shift screenshot
[2024-02-07 22:52:41] INFO: Take largest contentful paint screenshot
[2024-02-07 22:52:44] INFO: Step 01: Success
[2024-02-07 22:52:44] INFO: Start to measure SC01_T02_SelectCategoryLaptops
[2024-02-07 22:52:49] INFO: Take after page complete check screenshot
[2024-02-07 22:52:52] INFO: Take cumulative layout shift screenshot
[2024-02-07 22:52:54] INFO: Take largest contentful paint screenshot
[2024-02-07 22:52:56] INFO: Step 02: Category selected successfully
[2024-02-07 22:52:56] INFO: Start to measure SC01_T03_SelectProductLaptops
[2024-02-07 22:52:56] ERROR: Couldn't execute script named CUSTOM error:JavascriptError: javascript error: Cannot read properties of null (reading 'click')
  (Session info: chrome=121.0.6167.140)
[2024-02-07 22:52:56] ERROR: Could not find element by xpath //*[@id="tbodyid"]/div[1]/div/div/h4/b
[2024-02-07 22:52:56] ERROR: Step 03: Could not Select Product
[2024-02-07 22:52:56] INFO: https://www.demoblaze.com/# has been tested before within the same run, it will get an extra query parameter named browsertime_run. Make sure to use alias to keep track of the URLs
[2024-02-07 22:52:56] INFO: Take after page complete check screenshot
[2024-02-07 22:52:57] INFO: Take cumulative layout shift screenshot
[2024-02-07 22:52:59] INFO: Take largest contentful paint screenshot
[2024-02-07 22:53:02] INFO: https://www.demoblaze.com/ TTFB: 2.20s DOMContentLoaded: 7.46s firstPaint: 5.91s FCP: 5.91s LCP: 5.91s Load: 9.01s CLS:0.0155
[2024-02-07 22:53:02] INFO: https://www.demoblaze.com/# TTFB: 2.20s DOMContentLoaded: 7.46s firstPaint: 5.91s FCP: 5.91s LCP: 5.91s Load: 9.01s CLS:0.0155
[2024-02-07 22:53:02] INFO: https://www.demoblaze.com/#?browsertime_run=2 TTFB: 2.20s DOMContentLoaded: 7.46s firstPaint: 5.91s FCP: 5.91s LCP: 5.91s Load: 9.01s CLS:0.0155
[2024-02-07 22:53:02] ERROR: Could not add meta data to the HAR, miss page 4
[2024-02-07 22:53:02] ERROR: Could not add meta data to the HAR, miss page 5
[2024-02-07 22:53:02] INFO: https://www.demoblaze.com/ 26 requests, TTFB: 1.80s (σ406.00ms 22.6%), firstPaint: 3.96s (σ1.96s 49.5%), FCP: 3.96s (σ1.96s 49.5%), DOMContentLoaded: 5.62s (σ1.84s 32.7%), LCP: 4.27s (σ1.64s 38.4%), CLS: 0.0129 (σ0.00 20.0%), TBT: 0ms (σ0.00ms 0%), CPUBenchmark: 87ms (σ4.00ms 4.0%), Load: 7.40s (σ1.61s 21.7%) (2 runs)
[2024-02-07 22:53:02] INFO: https://www.demoblaze.com/# 4 requests, TTFB: 1.80s (σ406.00ms 22.6%), firstPaint: 3.96s (σ1.96s 49.5%), FCP: 3.96s (σ1.96s 49.5%), DOMContentLoaded: 5.62s (σ1.84s 32.7%), LCP: 4.27s (σ1.64s 38.4%), CLS: 0.0129 (σ0.00 20.0%), TBT: 0ms (σ0.00ms 0%), CPUBenchmark: 82ms (σ2.00ms 1.8%), Load: 7.40s (σ1.61s 21.7%) (2 runs)
[2024-02-07 22:53:02] INFO: https://www.demoblaze.com/#?browsertime_run=2 28 requests, TTFB: 1.80s (σ406.00ms 22.6%), firstPaint: 3.96s (σ1.96s 49.5%), FCP: 3.96s (σ1.96s 49.5%), DOMContentLoaded: 5.62s (σ1.84s 32.7%), LCP: 4.27s (σ1.64s 38.4%), CLS: 0.0129 (σ0.00 20.0%), TBT: 0ms (σ0.00ms 0%), CPUBenchmark: 87ms (σ8.00ms 8.7%), Load: 7.40s (σ1.61s 21.7%) (2 runs)
[2024-02-07 22:53:03] ERROR: Could not find the right index 4 for har for url https://www.demoblaze.com/#
[2024-02-07 22:53:03] ERROR: There are pages that misses data. This is in almost all cases caused by that you try to measure a page view but you do not navigate to a new page. Error: PageIndex out of range
    at module.exports.pickAPage (C:\Users\mjs\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\coach-core\lib\har\harCutter.js:18:11)
    at pickAPage (C:\Users\mjs\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\coach-core\lib\index.js:85:12)
    at BrowsertimePlugin.processMessage (file:///C:/Users/mjs/AppData/Roaming/npm/node_modules/sitespeed.io/lib/plugins/browsertime/index.js:282:29)
[2024-02-07 22:53:03] ERROR: Caught error from Browsertime Error: PageIndex out of range
    at module.exports.pickAPage (C:\Users\mjs\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\coach-core\lib\har\harCutter.js:18:11)
    at pickAPage (C:\Users\mjs\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\coach-core\lib\index.js:85:12)
    at BrowsertimePlugin.processMessage (file:///C:/Users/mjs/AppData/Roaming/npm/node_modules/sitespeed.io/lib/plugins/browsertime/index.js:518:33)
[2024-02-07 22:53:04] ERROR: Could not generate url/iteration/index, C:\Users\mjs\AppData\Roaming\npm\node_modules\sitespeed.io\lib\plugins\html\templates\url\thirdparty\index.pug:24
    22| h2 Third party
    23|
  > 24| - const thirdPartyWebVersion = pageInfo.data.coach.run ? pageInfo.data.coach.run.thirdPartyWebVersion : pageInfo.data.coach.pageSummary.thirdPartyWebVersion;
    25| p Third party requests categorised by
    26|     a(href='https://github.com/patrickhulce/third-party-web') Third party web
    27|     |  version #{thirdPartyWebVersion}.

Cannot read properties of undefined (reading 'run')
node:fs:2288
    validateStringAfterArrayBufferView(data, 'data');
    ^

TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
    at writeFile (node:fs:2288:5)
    at node:internal/util:443:21
    at new Promise (<anonymous>)
    at writeFile (node:internal/util:429:12)
    at write (file:///C:/Users/mjs/AppData/Roaming/npm/node_modules/sitespeed.io/lib/core/resultsStorage/storageManager.js:26:10)
    at file:///C:/Users/mjs/AppData/Roaming/npm/node_modules/sitespeed.io/lib/core/resultsStorage/storageManager.js:113:9 {
  code: 'ERR_INVALID_ARG_TYPE'
}

Node.js v21.2.0
soulgalore commented 5 months ago

Hi @Pearlking311 and thank you for the bug report! I also get error but I get a base HTML page generated. Do you add any other cli parameters when you run the test? Wanted to check so I know that I fix all things when I work on it. When I tested it I used: sitespeed.io test.cjs --multi

Pearlking311 commented 5 months ago

Hi @soulgalore, Sorry I missed to include it. I used: sitespeed.io --config D:\DemoblazeConfig.json --multi D:\test2.js DemoblazeConfig.json

soulgalore commented 5 months ago

I got a fix that I can toll out this weekend, but I also think we are missing something, like we should have something like stopAsError that we could use in a catch block so we don't collect metrics for a test that failed. I'll check if I can do something about it.

soulgalore commented 5 months ago

A fix for this was released last week + checkout stopAsError in https://www.sitespeed.io/documentation/sitespeed.io/scripting/tutorial-06-Error-handling.html