Open vikasmishra1973 opened 1 year ago
adding pre script step
09:55:34 [2023-05-22 16:55:34] INFO: Start of PreScript 09:55:34 [2023-05-22 16:55:34] INFO: Using url: https://creativecloud.adobe.com/cc?context=xxx 09:55:34 [2023-05-22 16:55:34] INFO: Navigating to url https://creativecloud.adobe.com/cc?context=xxx iteration 1 09:55:57 [2023-05-22 16:55:57] INFO: waiting for 10 seconds 09:55:57 [2023-05-22 16:55:57] INFO: susi email value entered is "xxxxx"
09:55:57 [2023-05-22 16:55:57] INFO: sleep for sometime before clicking Continue Button
09:56:13 [2023-05-22 16:56:13] INFO: Waiting for CCHome WebApp
09:56:14 [2023-05-22 16:56:14] INFO: End of PreScript
09:56:14 [2023-05-22 16:56:14] INFO: ****
09:56:14 [2023-05-22 16:56:14] INFO: Testing url https://creativecloud.adobe.com/cc iteration 1
09:56:31 [2023-05-22 16:56:31] INFO: Take after page complete check screenshot
09:56:32 [2023-05-22 16:56:32] INFO: Take cumulative layout shift screenshot
09:56:32 [2023-05-22 16:56:32] INFO: Take largest contentful paint screenshot
09:56:35 [2023-05-22 16:56:35] INFO: Get visual metrics from the video
09:56:40 [2023-05-22 16:56:40] ERROR: Could not add fullyLoaded metric to URL undefined, we have statistic for the URLs ["https://creativecloud.adobe.com/cc"]
09:56:40 [2023-05-22 16:56:40] ERROR: Could not add fullyLoaded metric to URL undefined, we have statistic for the URLs ["https://creativecloud.adobe.com/cc"]
09:56:40 [2023-05-22 16:56:40] ERROR: Caught error from Browsertime TypeError: Cannot read properties of undefined (reading 'response')
09:56:40 at getDocumentRequests (/usr/src/app/node_modules/browsertime/lib/support/har/index.js:33:18)
09:56:40 at getFinalURL (/usr/src/app/node_modules/browsertime/lib/support/har/index.js:39:20)
09:56:40 at Object.getMainDocumentTimings (/usr/src/app/node_modules/browsertime/lib/support/har/index.js:200:24)
09:56:40 at Engine.runByScript (/usr/src/app/node_modules/browsertime/lib/core/engine/index.js:301:31)
09:56:40 at async Object.analyzeUrl (/usr/src/app/lib/plugins/browsertime/analyzer.js:152:21)
09:56:40 at async Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:150:26)
09:56:40 [2023-05-22 16:56:40] INFO: Running tests using Chrome - 1 iteration(s)
09:56:40 [2023-05-22 16:56:40] INFO: **
09:56:40 [2023-05-22 16:56:40] INFO: Start of PreScript
09:56:41 [2023-05-22 16:56:41] INFO: Using url: https://creativecloud.adobe.com/cc?context=nnnn
09:56:41 [2023-05-22 16:56:41] INFO: Navigating to url https://creativecloud.adobe.com/cc?context=nnnn
iteration 1
09:57:01 [2023-05-22 16:57:01] INFO: waiting for 10 seconds
09:57:01 [2023-05-22 16:57:01] INFO: susi email value entered is "bbbb"
09:57:01 [2023-05-22 16:57:01] INFO: env user value entered is undefined
09:57:01 [2023-05-22 16:57:01] INFO: sleep for sometime before clicking Continue Button
09:57:17 [2023-05-22 16:57:17] INFO: Waiting for CCHome WebApp
09:57:18 [2023-05-22 16:57:18] INFO: End of PreScript
09:57:18 [2023-05-22 16:57:18] INFO: ****
09:57:18 [2023-05-22 16:57:18] INFO: Testing url https://creativecloud.adobe.com/cc/?addfeatures=adobe_home_m0 iteration 1
09:57:36 [2023-05-22 16:57:36] INFO: Take after page complete check screenshot
09:57:36 [2023-05-22 16:57:36] INFO: Take cumulative layout shift screenshot
09:57:37 [2023-05-22 16:57:37] INFO: Take largest contentful paint screenshot
09:57:40 [2023-05-22 16:57:40] INFO: Get visual metrics from the video
09:57:44 [2023-05-22 16:57:44] ERROR: Could not add fullyLoaded metric to URL undefined, we have statistic for the URLs ["https://creativecloud.adobe.com/cc/?addfeatures=adobe_home_m0"]
09:57:44 [2023-05-22 16:57:44] INFO: https://creativecloud.adobe.com/cc/?addfeatures=adobe_home_m0 21 requests, TTFB: 200ms, firstPaint: 734ms, firstVisualChange: 167ms, FCP: 734ms, DOMContentLoaded: 596ms, LCP: 1.79s, CLS: 0.0546, TBT: 1.05s, Load: 788ms, speedIndex: 1.65s, visualComplete85: 3.43s, lastVisualChange: 4.87s
09:57:45 [2023-05-22 16:57:45] ERROR: TypeError: Cannot convert undefined or null to object
09:57:45 at Function.keys (
Hi @vikasmishra1973 thanks for creating the issue! Can you try with the latest released version of sitespeed.io 27.6.4 (the version you are using are almost one year old), then it's easier for me to reproduce, try to fix it. Thanks!
13:45:45 Pulling sitespeed.io (sitespeedio/sitespeed.io:27.6.4)...
13:45:47 27.6.4: Pulling from sitespeedio/sitespeed.io
13:46:47 Digest: sha256:7e69c15863dd825699f3a6a020427ecdbcfb48d5c77e15ccae5d12413c1ca513
13:46:47 Status: Downloaded newer image for sitespeedio/sitespeed.io:27.6.4
13:47:25 Google Chrome 113.0.5672.92
13:47:26 Mozilla Firefox 113.0
13:47:28 Microsoft Edge 113.0.1774.42
13:47:39 [2023-05-22 20:47:39] INFO: Versions OS: linux 5.4.0-74-generic nodejs: v18.16.0 sitespeed.io: 27.6.4 browsertime: 17.10.1 coach: 7.1.3
13:47:39 (node:41) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
13:47:39
13:47:39 Please migrate your code to use AWS SDK for JavaScript (v3).
13:47:39 For more information, check the migration guide at https://a.co/7PzMCcy
13:47:39 (Use node --trace-warnings ...
to show where the warning was created)
13:47:39 [2023-05-22 20:47:39] INFO: Running tests using Chrome - 1 iteration(s)
13:47:45 [2023-05-22 20:47:45] INFO: **
13:47:45 [2023-05-22 20:47:45] INFO: Start of PreScript
13:47:45 [2023-05-22 20:47:45] INFO: Using url: https://creativecloud.adobe.com/cc?context=no_cc_redirect
13:47:45 [2023-05-22 20:47:45] INFO: Navigating to url https://creativecloud.adobe.com/cc?context=no_cc_redirect iteration 1
13:48:05 [2023-05-22 20:48:05] INFO: waiting for 10 seconds
13:48:05 [2023-05-22 20:48:05] INFO: susi email value entered is "vvvvv"
13:48:05 [2023-05-22 20:48:05] INFO: sleep for sometime before clicking Continue Button
13:48:20 [2023-05-22 20:48:20] INFO: Waiting for CCHome WebApp
13:48:22 [2023-05-22 20:48:22] INFO: End of PreScript
13:48:22 [2023-05-22 20:48:22] INFO: ****
13:48:22 [2023-05-22 20:48:22] INFO: Testing url https://creativecloud.adobe.com/cc iteration 1
13:48:50 [2023-05-22 20:48:50] INFO: Take after page complete check screenshot
13:48:51 [2023-05-22 20:48:51] INFO: Take cumulative layout shift screenshot
13:48:52 [2023-05-22 20:48:52] INFO: Take largest contentful paint screenshot
13:48:55 [2023-05-22 20:48:55] INFO: Use the visual metrics portable script
13:48:55 [2023-05-22 20:48:55] INFO: Get visual metrics from the video
13:49:49 [2023-05-22 20:49:49] ERROR: Could not add fullyLoaded metric to URL undefined, we have statistic for the URLs ["https://creativecloud.adobe.com/cc"]
13:49:49 [2023-05-22 20:49:49] INFO: https://creativecloud.adobe.com/cc 7 requests, TTFB: 374ms, firstPaint: 905ms, firstVisualChange: 234ms, FCP: 905ms, DOMContentLoaded: 728ms, LCP: 1.88s, CLS: 0.0448, TBT: 1.76s, CPUBenchmark: 77ms, Load: 5.29s, speedIndex: 1.97s, visualComplete85: 5.13s, lastVisualChange: 5.87s
13:49:50 [2023-05-22 20:49:50] ERROR: TypeError: Cannot convert undefined or null to object
13:49:50 at Function.keys (
@soulgalore , Hi Peter, any pointer for me?
Hi @vikasmishra1973 thanks for updating the issue. Hmm, I can see in the log that there's some kind of mismatch going on between the URLs (like somehow the data is attached to the wrong URL or something). Do you know where https://creativecloud.adobe.com/undefined comes? Undefined seems like something is wrong.
Looking at the output, it looks like the first navigation to the page do not happen, if you look in the browser, are the browser already on that page?
13:48:22 [2023-05-22 20:48:22] INFO: Testing url https://creativecloud.adobe.com/cc iteration 1
13:48:50 [2023-05-22 20:48:50] INFO: Take after page complete check screenshot
13:48:51 [2023-05-22 20:48:51] INFO: Take cumulative layout shift screenshot
13:48:52 [2023-05-22 20:48:52] INFO: Take largest contentful paint screenshot
13:48:55 [2023-05-22 20:48:55] INFO: Use the visual metrics portable script
13:48:55 [2023-05-22 20:48:55] INFO: Get visual metrics from the video
13:49:49 [2023-05-22 20:49:49] ERROR: Could not add fullyLoaded metric to URL undefined
Best Peter
Hi, I ran it locally on my mac with just one url:
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:27.6.4 --graphite.host=host.docker.internal https://creativecloud.adobe.com/cc --browsertime.preScript ./ccweb-susi.js --slug yourTestName --graphite.addSlugToKey true
Google Chrome 113.0.5672.92
Mozilla Firefox 113.0
Microsoft Edge 113.0.1774.42
[2023-05-25 16:58:20] INFO: Versions OS: linux 5.15.49-linuxkit nodejs: v18.16.0 sitespeed.io: 27.6.4 browsertime: 17.10.1 coach: 7.1.3
[2023-05-25 16:58:21] INFO: Running tests using Chrome - 3 iteration(s)
[2023-05-25 16:58:21] INFO: **
[2023-05-25 16:58:21] INFO: Start of PreScript
[2023-05-25 16:58:21] INFO: Using url: https://creativecloud.adobe.com/cc
[2023-05-25 16:58:21] INFO: Navigating to url https://creativecloud.adobe.com/cc iteration 1
[2023-05-25 16:58:41] INFO: waiting for 10 seconds
[2023-05-25 16:58:41] INFO: susi email value entered is "tl+HomeMonitors@adobetest.com"
[2023-05-25 16:58:41] INFO: sleep for sometime before clicking Continue Button
[2023-05-25 16:58:57] INFO: Waiting for CCHome WebApp
[2023-05-25 16:58:59] INFO: End of PreScript
[2023-05-25 16:58:59] INFO: ****
[2023-05-25 16:58:59] INFO: Testing url https://creativecloud.adobe.com/cc iteration 1
[2023-05-25 16:59:13] INFO: Take after page complete check screenshot
[2023-05-25 16:59:14] INFO: Take cumulative layout shift screenshot
[2023-05-25 16:59:14] INFO: Take largest contentful paint screenshot
[2023-05-25 16:59:17] INFO: Use the visual metrics portable script
[2023-05-25 16:59:17] INFO: Get visual metrics from the video
[2023-05-25 16:59:22] INFO: https://creativecloud.adobe.com/cc TTFB: 29ms DOMContentLoaded: 486ms firstPaint: 560ms FCP: 560ms LCP: 2.98s Load: 3.50s TBT: 829ms CLS:0.0808
[2023-05-25 16:59:22] INFO: VisualMetrics: FirstVisualChange: 300ms SpeedIndex: 1.81s VisualComplete85: 3.10s LastVisualChange: 4.13s
[2023-05-25 16:59:23] INFO: **
[2023-05-25 16:59:23] INFO: Start of PreScript
[2023-05-25 16:59:23] INFO: Using url: https://creativecloud.adobe.com/cc
[2023-05-25 16:59:23] INFO: Navigating to url https://creativecloud.adobe.com/cc iteration 2
[2023-05-25 16:59:43] INFO: waiting for 10 seconds
[2023-05-25 16:59:43] INFO: susi email value entered is "tl+HomeMonitors@adobetest.com"
[2023-05-25 16:59:43] INFO: sleep for sometime before clicking Continue Button
[2023-05-25 16:59:58] INFO: Waiting for CCHome WebApp
[2023-05-25 17:00:00] INFO: End of PreScript
[2023-05-25 17:00:00] INFO: ****
[2023-05-25 17:00:00] INFO: Testing url https://creativecloud.adobe.com/cc iteration 2
[2023-05-25 17:00:14] INFO: Take after page complete check screenshot
[2023-05-25 17:00:15] INFO: Take cumulative layout shift screenshot
[2023-05-25 17:00:15] INFO: Take largest contentful paint screenshot
[2023-05-25 17:00:18] INFO: Use the visual metrics portable script
[2023-05-25 17:00:18] INFO: Get visual metrics from the video
[2023-05-25 17:00:23] INFO: https://creativecloud.adobe.com/cc TTFB: 3ms DOMContentLoaded: 671ms firstPaint: 750ms FCP: 750ms LCP: 3.63s Load: 3.56s TBT: 842ms CLS:0.0768
[2023-05-25 17:00:23] INFO: VisualMetrics: FirstVisualChange: 100ms SpeedIndex: 2.27s VisualComplete85: 3.33s LastVisualChange: 4.37s
[2023-05-25 17:00:23] INFO: **
[2023-05-25 17:00:23] INFO: Start of PreScript
[2023-05-25 17:00:23] INFO: Using url: https://creativecloud.adobe.com/cc
[2023-05-25 17:00:23] INFO: Navigating to url https://creativecloud.adobe.com/cc iteration 3
[2023-05-25 17:00:43] INFO: waiting for 10 seconds
[2023-05-25 17:00:43] INFO: susi email value entered is "tl+HomeMonitors@adobetest.com"
[2023-05-25 17:00:43] INFO: sleep for sometime before clicking Continue Button
[2023-05-25 17:00:59] INFO: Waiting for CCHome WebApp
[2023-05-25 17:01:01] INFO: End of PreScript
[2023-05-25 17:01:01] INFO: ****
[2023-05-25 17:01:01] INFO: Testing url https://creativecloud.adobe.com/cc iteration 3
[2023-05-25 17:01:14] INFO: Take after page complete check screenshot
[2023-05-25 17:01:15] INFO: Take cumulative layout shift screenshot
[2023-05-25 17:01:15] INFO: Take largest contentful paint screenshot
[2023-05-25 17:01:18] INFO: Use the visual metrics portable script
[2023-05-25 17:01:18] INFO: Get visual metrics from the video
[2023-05-25 17:01:24] INFO: https://creativecloud.adobe.com/cc TTFB: 5ms DOMContentLoaded: 256ms firstPaint: 330ms FCP: 330ms LCP: 2.37s Load: 3.40s TBT: 759ms CLS:0.0704
[2023-05-25 17:01:24] INFO: VisualMetrics: FirstVisualChange: 33ms SpeedIndex: 1.32s VisualComplete85: 1.20s LastVisualChange: 3.80s
[2023-05-25 17:01:24] ERROR: Could not add fullyLoaded metric to URL undefined, we have statistic for the URLs ["https://creativecloud.adobe.com/cc"]
[2023-05-25 17:01:24] ERROR: Could not add fullyLoaded metric to URL undefined, we have statistic for the URLs ["https://creativecloud.adobe.com/cc"]
[2023-05-25 17:01:24] ERROR: Could not add fullyLoaded metric to URL undefined, we have statistic for the URLs ["https://creativecloud.adobe.com/cc"]
[2023-05-25 17:01:24] INFO: https://creativecloud.adobe.com/cc 73 requests, TTFB: 5ms (σ12.00ms 95.8%), firstPaint: 560ms (σ172.00ms 31.4%), firstVisualChange: 100ms (σ113.00ms 78.6%), FCP: 560ms (σ172.00ms 31.4%), DOMContentLoaded: 486ms (σ170.00ms 36.0%), LCP: 2.98s (σ515.00ms 17.2%), CLS: 0.0768 (σ0.00 5.6%), TBT: 829ms (σ36.00ms 4.5%), CPUBenchmark: 49ms (σ1.00ms 2.8%), Load: 3.50s (σ65.00ms 1.9%), speedIndex: 1.81s (σ387.00ms 21.5%), visualComplete85: 3.10s (σ955.00ms 37.5%), lastVisualChange: 4.13s (σ233.00ms 5.7%) (3 runs)
[2023-05-25 17:01:25] ERROR: TypeError: Cannot convert undefined or null to object
at Function.keys (
I'm having the same issue when using scripts to test a page.
The issue seems to be related to the documentation link; where it states that
Browsertime/sitespeed.io will pick up the next URL and measure that.
when my script is of the form:
test6.js: |
module.exports = async function(context, commands) {
const GAMEURL = 'https://someurl'
const PLAYCSSSELECTOR = '.play-real-button'
context.log.info('The URL will be');
context.log.info(GAMEURL);
commands.measure.start('MyAlias');
await commands.navigate(GAMEURL);
await commands.click.bySelectorAndWait(PLAYCSSSELECTOR);
return commands.measure.stop()
};
I get the errors above, namely:
On the other hand, if my test script is of the form (same as documentation):
test6.js: |
module.exports = async function(context, commands) {
const GAMEURL = 'https://someurl'
const PLAYCSSSELECTOR = '.play-real-button'
context.log.info('The URL will be');
context.log.info(GAMEURL);
await commands.navigate(GAMEURL);
commands.measure.start('MyAlias');
await commands.click.bySelectorAndWait(PLAYCSSSELECTOR);
return commands.measure.stop()
};
errors are gone... but the stop contidion on my check fails (it is a slow javascript animation that loads in about 11 seconds, and the progress bar gets stuck on about 4 seconds, causing the check to end prematurely).
Not sure if (1) it shares the root cause for this case and (2) if there is a way to define the 'url' parameter to be consumed by the browsertime engine when tests are based on script instead of option url? kindly note that the preferred command commands.measure.start('https://someurl'); does not suffice on my use case, as I need to click on a CSS element loaded on the measured page and continues the the full load time measurement until the load of the 'seleted/clicked' element completes.
the docker run, is pretty much the same, I just call a preScript (for login) and the test script using the multi option
# Scripts
- "--preScript"
- "/tmp/auth/logonscript.mjs"
- "/tmp/scripts/test6.js"
- "--multi"
Note: added here as I share the output errors, save for domain names, kindly let me konw if this comment needs to be deleted and added to a different bug ticket. Best,
Hi @PedroMSantosD I think your case is another issue. If I understand correctly, the tests stops too early in one of the cases? By default when you use a xxxWait command Browsertime waits on the onLoad event to happen. You can try run your test and add --browsertime.pageCompleteCheckNetworkIdle true
instead. That will wait for 5 seconds of inactivity on the network log (meaning the unload event can fire but you still got Ajax requests going on). Can you try that and see if you get a different result?
Hi Peter, for the pageCompletCheck condition, I do get different conditions using either "--pageCompleteCheckInactivity" or "--pageCompleteCheckNetworkIdle" or (default) end condition. the pageCompleteCheckInactivity gets a proper screenshot for the report summary, but the filstrip does not shows this transitions. This part is fully unrelated to the reported bug, and it is more of my use case and end condition (thanks).
That being said, if my test scripts uses the commands.navigate(URL); command after the measure.start(alias) the errors are replicated in full. whereas if my test script uses the commands.navigate(URL) before the measure.start(alias), all errors listed here are gone.
Will drop both cases off this ticket for sake of clarity. Thanks a lot!!
Have you read the documentation?
URL
https://creativecloud.adobe.com/cc
What are you trying to accomplish
Logging on the url using pre script an d checking an element as to how much time it takes to load
What browser did you use?
Chrome
How to reproduce
Relevant log output