search

sitespeedio / sitespeed.io

sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.
https://www.sitespeed.io/
MIT License
4.73k stars 600 forks source link

Sitespeed GPSI plugin with application on private network #3979

Closed dawsze closed 8 months ago

dawsze commented 11 months ago

Your question

I guess it's more question than a bug report but I'm trying to test application on private network. I'm mostly intrested in finding out INP metric from GPSI plugin. When i ran command: 'docker run --shm-size=1g --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:29.7.0-plus1 -n 1 ${PAGE_ADDRESS} --plugins.remove /lighthouse' no matter I pass --gpsi.key or not, the GPSI page at report is not shown.

Does it even possible to run gpsi inside private network? If so, please tell me what I'm doing wrong because, i could't find any informations bout that in google/yours repo.

Environment: Ubuntu 22.04.2 LTS [Virtualbox VM] Docker 24.0.6, build ed223bc

Log from cmd:

Google Chrome 117.0.5938.92 Mozilla Firefox 118.0 Microsoft Edge 117.0.2045.40 [2023-10-12 12:59:57] INFO: Versions OS: linux 6.2.0-34-generic nodejs: v18.18.0 sitespeed.io: 29.7.0 browsertime: 17.16.0 coach: 8.0.2 [2023-10-12 13:00:00] INFO: Using default Lighthouse configuration for desktop [2023-10-12 13:00:00] INFO: Will run Lighthouse tests after Browsertime has finished [2023-10-12 13:00:00] INFO: Sending url page_address to test on Page Speed Insights [2023-10-12 13:00:00] INFO: Running tests using Chrome - 1 iteration(s) [2023-10-12 13:00:01] INFO: Testing url page_address iteration 1 [2023-10-12 13:00:13] INFO: Take after page complete check screenshot [2023-10-12 13:00:14] INFO: Take cumulative layout shift screenshot [2023-10-12 13:00:15] INFO: Take largest contentful paint screenshot [2023-10-12 13:00:18] INFO: Use the visual metrics portable script [2023-10-12 13:00:18] INFO: Get visual metrics from the video [2023-10-12 13:00:19] ERROR: Error: Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_TIMED_OUT) at Gaxios._request (/gpsi/node_modules/gaxios/build/src/gaxios.js:85:23) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) [2023-10-12 13:00:24] INFO: page_address 108 requests, TTFB: 271ms, firstPaint: 461ms, firstVisualChange: 466ms, FCP: 461ms, DOMContentLoaded: 855ms, LCP: 513ms, CLS: 0.0028, TBT: 607ms, CPUBenchmark: 55ms, Load: 2.19s, speedIndex: 1.95s, visualComplete85: 5.70s, lastVisualChange: 6.03s [2023-10-12 13:00:24] INFO: Start collecting Lighthouse result for page_address [2023-10-12 13:00:43] INFO: HTML stored in /sitespeed.io/sitespeed-result/page_address/2023-10-12-12-59-57 Errors while running: Rejected { "uuid": "04e79a0f-5815-490b-a434-bc1e7c47e90f", "type": "url", "timestamp": "2023-10-12T13:00:00+00:00", "source": "url-reader", "data": "{...}", "url": "page_address", "group": "page_address" } for plugin: gpsi, url: page_address {"response":{"config":{"url":"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?strategy=desktop&url=https%3A%2F%2Fpage_address%2F&nokey=true&category=seo&category=accessibility&category=pwa&category=best-practices&category=performance","method":"GET","headers":{"x-goog-api-client":"gdcl/3.2.2 gl-node/18.18.0 auth/5.10.1","Accept-Encoding":"gzip","User-Agent":"google-api-nodejs-client/3.2.2 (gzip)","Accept":"application/json"},"params":{"strategy":"desktop","url":"page_address","nokey":true,"category":["seo","accessibility","pwa","best-practices","performance"]},"retry":true,"responseType":"json","retryConfig":{"currentRetryAttempt":0,"retry":3,"httpMethodsToRetry":["GET","HEAD","PUT","OPTIONS","DELETE"],"noResponseRetries":2,"statusCodesToRetry":[[100,199],[429,429],[500,599]]}},"data":{"error":{"code":400,"message":"Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_TIMED_OUT)","errors":[{"message":"Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_TIMED_OUT)","domain":"lighthouse","reason":"lighthouseUserError"}]}},"headers":{"alt-svc":"h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000","cache-control":"private","connection":"close","content-encoding":"gzip","content-type":"application/json; charset=UTF-8","date":"Thu, 12 Oct 2023 13:00:19 GMT","server":"ESF","transfer-encoding":"chunked","vary":"Origin, X-Origin, Referer","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","x-xss-protection":"0"},"status":400,"statusText":"Bad Request","request":{"responseURL":"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?strategy=desktop&url=https%3A%2F%2Fpage_address%2F&nokey=true&category=seo&category=accessibility&category=pwa&category=best-practices&category=performance"}},"config":{"url":"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?strategy=desktop&url=https%3A%2F%2Fpage_address%2F&nokey=true&category=seo&category=accessibility&category=pwa&category=best-practices&category=performance","method":"GET","headers":{"x-goog-api-client":"gdcl/3.2.2 gl-node/18.18.0 auth/5.10.1","Accept-Encoding":"gzip","User-Agent":"google-api-nodejs-client/3.2.2 (gzip)","Accept":"application/json"},"params":{"strategy":"desktop","url":"page_address","nokey":true,"category":["seo","accessibility","pwa","best-practices","performance"]},"retry":true,"responseType":"json","retryConfig":{"currentRetryAttempt":0,"retry":3,"httpMethodsToRetry":["GET","HEAD","PUT","OPTIONS","DELETE"],"noResponseRetries":2,"statusCodesToRetry":[[100,199],[429,429],[500,599]]}},"code":400,"errors":[{"message":"Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_TIMED_OUT)","domain":"lighthouse","reason":"lighthouseUserError"}]} Error: Errors while running: Rejected { "uuid": "04e79a0f-5815-490b-a434-bc1e7c47e90f", "type": "url", "timestamp": "2023-10-12T13:00:00+00:00", "source": "url-reader", "data": "{...}", "url": "page_address", "group": "page_address" } for plugin: gpsi, url: page_address {"response":{"config":{"url":"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?strategy=desktop&url=https%3A%2F%2Fpage_address%2F&nokey=true&category=seo&category=accessibility&category=pwa&category=best-practices&category=performance","method":"GET","headers":{"x-goog-api-client":"gdcl/3.2.2 gl-node/18.18.0 auth/5.10.1","Accept-Encoding":"gzip","User-Agent":"google-api-nodejs-client/3.2.2 (gzip)","Accept":"application/json"},"params":{"strategy":"desktop","url":"page_address","nokey":true,"category":["seo","accessibility","pwa","best-practices","performance"]},"retry":true,"responseType":"json","retryConfig":{"currentRetryAttempt":0,"retry":3,"httpMethodsToRetry":["GET","HEAD","PUT","OPTIONS","DELETE"],"noResponseRetries":2,"statusCodesToRetry":[[100,199],[429,429],[500,599]]}},"data":{"error":{"code":400,"message":"Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_TIMED_OUT)","errors":[{"message":"Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_TIMED_OUT)","domain":"lighthouse","reason":"lighthouseUserError"}]}},"headers":{"alt-svc":"h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000","cache-control":"private","connection":"close","content-encoding":"gzip","content-type":"application/json; charset=UTF-8","date":"Thu, 12 Oct 2023 13:00:19 GMT","server":"ESF","transfer-encoding":"chunked","vary":"Origin, X-Origin, Referer","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","x-xss-protection":"0"},"status":400,"statusText":"Bad Request","request":{"responseURL":"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?strategy=desktop&url=https%3A%2F%2Fpage_address%2F&nokey=true&category=seo&category=accessibility&category=pwa&category=best-practices&category=performance"}},"config":{"url":"https://www.googleapis.com/pagespeedonline/v5/runPagespeed?strategy=desktop&url=https%3A%2F%2Fpage_address%2F&nokey=true&category=seo&category=accessibility&category=pwa&category=best-practices&category=performance","method":"GET","headers":{"x-goog-api-client":"gdcl/3.2.2 gl-node/18.18.0 auth/5.10.1","Accept-Encoding":"gzip","User-Agent":"google-api-nodejs-client/3.2.2 (gzip)","Accept":"application/json"},"params":{"strategy":"desktop","url":"page_address","nokey":true,"category":["seo","accessibility","pwa","best-practices","performance"]},"retry":true,"responseType":"json","retryConfig":{"currentRetryAttempt":0,"retry":3,"httpMethodsToRetry":["GET","HEAD","PUT","OPTIONS","DELETE"],"noResponseRetries":2,"statusCodesToRetry":[[100,199],[429,429],[500,599]]}},"code":400,"errors":[{"message":"Lighthouse returned error: FAILED_DOCUMENT_REQUEST. Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Details: net::ERR_TIMED_OUT)","domain":"lighthouse","reason":"lighthouseUserError"}]} at start (file:///usr/src/app/bin/sitespeed.js:167:15) at processTicksAndRejections (node:internal/process/task_queues:95:5) at runNextTicks (node:internal/process/task_queues:64:3) at process.processImmediate (node:internal/timers:447:9) at async file:///usr/src/app/bin/sitespeed.js:178:1

polarnik commented 10 months ago

The gpsi plugin is a cloud version of the lighthouse plugin. The gpsi plugin needs to have a web access to https://pagespeed.web.dev/. You can use the lighthouse plugin without any additional web access rules.

polarnik commented 10 months ago

The lighthouse plugin is better than the gpsi plugin for slow pages. Because the lighthouse plugin has a config file. And a test developer can increase timeout settings. It is impossible for https://pagespeed.web.dev/ and the gpsi

The config example with maxWaitForFcp and maxWaitForLoad:

module.exports = {
    "extends": "lighthouse:default",
    "settings": {
        "maxWaitForFcp": 60000,
        "maxWaitForLoad": 350000,
        "formFactor": "desktop",
        "throttling": {
            "rttMs": 40,
            "throughputKbps": 10240,
            "cpuSlowdownMultiplier": 1,
            "requestLatencyMs": 0,
            "downloadThroughputKbps": 0,
            "uploadThroughputKbps": 0
        },
        "screenEmulation": {
            "mobile": false,
            "width": 1350,
            "height": 940,
            "deviceScaleFactor": 1,
            "disabled": false
        },
        "emulatedUserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.5845.96  crawler/1.0",
        "skipAudits": [
        ]
    }
};

and the SiteSpeed argument is needed:

--lighthouse.config "/config/lighthouse.config.js"
soulgalore commented 8 months ago

Please follow @polarnik advice.