search

sitespeedio / sitespeed.io

sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.
https://www.sitespeed.io/
MIT License
4.75k stars 602 forks source link

sitespeed.io dashboard (Graphite and Grafana) on azure. #3832

Closed AlanCodega closed 1 year ago

AlanCodega commented 1 year ago

Have you read the documentation?

URL

https://google.com

What are you trying to accomplish

Hi All, i've setup as Azure Container group: image Grafana and graphite, deployed with this azure template (i transform basically the docker-compose file into azure template, adding permanent azure storage):

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {},
  "variables": {},
  "resources": [
    {
      "type": "Microsoft.ContainerInstance/containerGroups",
      "apiVersion": "2021-03-01",
      "name": "grafana-graphite-template",
      "location": "[resourceGroup().location]",
      "properties": {
        "containers": [
          {
            "name": "grafana",
            "properties": {
              "image": "grafana/grafana:9.4.7",
              "ports": [
                {
                  "protocol": "TCP",
                  "port": 3000
                }
              ],
              "environmentVariables": [
                {
                  "name": "GF_SECURITY_ADMIN_PASSWORD",
                  "value": "user"
                },
                {
                  "name": "GF_SECURITY_ADMIN_USER",
                  "value": "user"
                },
                {
                  "name": "GF_AUTH_ANONYMOUS_ENABLED",
                  "value": "true"
                },
                {
                  "name": "GF_USERS_ALLOW_SIGN_UP",
                  "value": "true"
                },
                {
                  "name": "GF_USERS_ALLOW_ORG_CREATE",
                  "value": "false"
                },
                {
                  "name": "GF_INSTALL_PLUGINS",
                  "value": "grafana-piechart-panel"
                }
              ],
              "resources": {
                "requests": {
                  "cpu": 1,
                  "memoryInGb": 1.5
                }
              },
              "volumeMounts": [
                {
                  "name": "grafana",
                  "mountPath": "/var/lib/grafana"
                }
              ]
            }
          },
          {
            "name": "graphite",
            "properties": {
              "image": "sitespeedio/graphite:1.1.10-3",
              "ports": [
                {
                  "protocol": "TCP",
                  "port": 2003
                },
                {
                  "protocol": "TCP",
                  "port": 8080
                }
              ],
              "resources": {
                "requests": {
                  "cpu": 1,
                  "memoryInGb": 1.5
                }
              },
              "volumeMounts": [
                {
                  "name": "whisper",
                  "mountPath": "/opt/graphite/storage/whisper"
                }
              ]
            }
          }
        ],
        "osType": "Linux",
        "restartPolicy": "Always",
        "ipAddress": {
          "type": "Public",
          "dnsNameLabel": "grafana",
          "ports": [
            {
              "protocol": "TCP",
              "port": 3000
            },
            {
              "protocol": "TCP",
              "port": 2003
            },
            {
              "protocol": "TCP",
              "port": 8080
            }
          ]
        },
        "volumes": [
          {
            "name": "grafana",
            "azureFile": {
              "shareName": "grafana-data-compose",
              "storageAccountName": "username",
              "storageAccountKey": "Pxyxyxyxyxyxyxyxyxyxy"
            }
          },
          {
            "name": "whisper",
            "azureFile": {
              "shareName": "grafana-data-compose-whisper",
              "storageAccountName": "username",
              "storageAccountKey": "yxyxyxyxyxyxyxyxyxyxyxyx"
            }
          }
        ]
      }
    }
  ]
}

deployed correctly, all the ports are opened, and I can open with browser Graphite and Grafana ui.

For cost saving, I'm using sitespeed.io container locally on my docker with this command:

docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:27.3.0 https://google.com --graphite.host cirrusqa.westeurope.azurecontainer.io --outputFolder output --slug DEV --graphite.addSlugToKey true

but when opening grafana UI from browser, with the already pre-maded dashboard (https://github.com/sitespeedio/grafana-bootstrap-docker/blob/main/dashboards/graphite/Leaderboard.json) i see only "NO DATA".

Any idea or suggestion what can be?

Many thanks in advance!

What browser did you use?

Chrome

How to reproduce

docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:27.3.0 https://google.com --graphite.host cirrusqa.westeurope.azurecontainer.io --outputFolder output --slug DEV --graphite.addSlugToKey
true

Relevant log output

ubuntu@DESKTOP-SBSBK56:~$  docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:27.3.0 https://google.com --graphite.host cirrusqa.westeurope.azurecontainer.io --outputFolder output --slug DEV --graphite.addSlugToKey
true
Google Chrome 112.0.5615.49
Mozilla Firefox 112.0
Microsoft Edge 112.0.1722.39
[2023-05-04 05:50:57] INFO: Versions OS: linux 5.15.90.1-microsoft-standard-WSL2 nodejs: v18.14.2 sitespeed.io: 27.3.0 browsertime: 17.8.0 coach: 7.1.3
[2023-05-04 05:50:57] INFO: Running tests using Chrome - 3 iteration(s)
[2023-05-04 05:50:58] INFO: Testing url https://google.com iteration 1
[2023-05-04 05:51:06] INFO: Take after page complete check screenshot
[2023-05-04 05:51:07] INFO: Take cumulative layout shift screenshot
[2023-05-04 05:51:08] INFO: Take largest contentful paint screenshot
[2023-05-04 05:51:11] INFO: Use the visual metrics portable script
[2023-05-04 05:51:11] INFO: Get visual metrics from the video
[2023-05-04 05:51:12] INFO: https://google.com TTFB: 73ms DOMContentLoaded: 233ms firstPaint: 135ms FCP: 135ms LCP: 199ms Load: 234ms CLS:0.0001
[2023-05-04 05:51:12] INFO: VisualMetrics: FirstVisualChange: 200ms SpeedIndex: 1.30s VisualComplete85: 1.30s LastVisualChange: 1.43s
[2023-05-04 05:51:13] INFO: Testing url https://google.com iteration 2
[2023-05-04 05:51:25] INFO: Take after page complete check screenshot
[2023-05-04 05:51:25] INFO: Take cumulative layout shift screenshot
[2023-05-04 05:51:26] INFO: Take largest contentful paint screenshot
[2023-05-04 05:51:29] INFO: Use the visual metrics portable script
[2023-05-04 05:51:29] INFO: Get visual metrics from the video
[2023-05-04 05:51:31] INFO: https://google.com TTFB: 1.84s DOMContentLoaded: 2.19s firstPaint: 2.05s FCP: 2.05s LCP: 2.11s Load: 2.19s CLS:0.0001
[2023-05-04 05:51:31] INFO: VisualMetrics: FirstVisualChange: 833ms SpeedIndex: 4.18s VisualComplete85: 4.20s LastVisualChange: 4.33s
[2023-05-04 05:51:31] INFO: Testing url https://google.com iteration 3
[2023-05-04 05:51:44] INFO: Take after page complete check screenshot
[2023-05-04 05:51:45] INFO: Take cumulative layout shift screenshot
[2023-05-04 05:51:45] INFO: Take largest contentful paint screenshot
[2023-05-04 05:51:48] INFO: Use the visual metrics portable script
[2023-05-04 05:51:48] INFO: Get visual metrics from the video
[2023-05-04 05:51:51] INFO: https://google.com TTFB: 1.34s DOMContentLoaded: 1.85s firstPaint: 1.70s FCP: 1.70s LCP: 1.79s Load: 1.85s CLS:0.0001
[2023-05-04 05:51:51] INFO: VisualMetrics: FirstVisualChange: 167ms SpeedIndex: 4.97s VisualComplete85: 5.00s LastVisualChange: 5.17s
[2023-05-04 05:51:51] INFO: https://google.com 34 requests, TTFB: 1.34s (σ743.00ms 68.6%), firstPaint: 1.70s (σ833.00ms 64.3%), firstVisualChange: 200ms (σ306.00ms 76.6%), FCP: 1.70s (σ833.00ms 64.3%), DOMContentLoaded: 1.85s (σ854.00ms 60.0%), LCP: 1.79s (σ837.00ms 61.2%), CLS: 0.0001 (σ0.00 0%), TBT: 0ms (σ0.00ms 0%), CPUBenchmark: 40ms (σ0.00ms 0%), Load: 1.85s (σ854.00ms 59.9%), speedIndex: 4.18s (σ1.58s 45.3%), visualComplete85: 4.20s (σ1.59s 45.4%), lastVisualChange: 4.33s (σ1.60s 43.9%) (3 runs)
[2023-05-04 05:51:52] INFO: HTML stored in /sitespeed.io/output
AlanCodega commented 1 year ago

with the command: docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:27.3.0 https://google.com --graphite.host cirrusqa.westeurope.azurecontainer.io --outputFolder output --slug google --graphite.addSlugToKey

I'm able to find in Grafana the testslug and the domain, but no data: image

output from shell:

(ubuntu@DESKTOP-SBSBK56:~$  docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:27.3.0 https://google.com --graphite.host cirrusqa.westeurope.azurecontainer.io --outputFolder output --slug google --graphite.addSlugToKey       Google Chrome 112.0.5615.49
Mozilla Firefox 112.0
Microsoft Edge 112.0.1722.39
[2023-05-04 07:24:04] INFO: Versions OS: linux 5.15.90.1-microsoft-standard-WSL2 nodejs: v18.14.2 sitespeed.io: 27.3.0 browsertime: 17.8.0 coach: 7.1.3
[2023-05-04 07:24:05] INFO: Running tests using Chrome - 3 iteration(s)
[2023-05-04 07:24:06] INFO: Testing url https://google.com iteration 1
[2023-05-04 07:24:14] INFO: Take after page complete check screenshot
[2023-05-04 07:24:14] INFO: Take cumulative layout shift screenshot
[2023-05-04 07:24:15] INFO: Take largest contentful paint screenshot
[2023-05-04 07:24:17] INFO: Use the visual metrics portable script
[2023-05-04 07:24:17] INFO: Get visual metrics from the video
[2023-05-04 07:24:19] INFO: https://google.com TTFB: 252ms DOMContentLoaded: 406ms firstPaint: 418ms FCP: 418ms LCP: 418ms Load: 510ms
[2023-05-04 07:24:19] INFO: VisualMetrics: FirstVisualChange: 434ms SpeedIndex: 434ms VisualComplete85: 434ms LastVisualChange: 467ms
[2023-05-04 07:24:19] INFO: Testing url https://google.com iteration 2
[2023-05-04 07:24:27] INFO: Take after page complete check screenshot
[2023-05-04 07:24:28] INFO: Take cumulative layout shift screenshot
[2023-05-04 07:24:28] INFO: Take largest contentful paint screenshot
[2023-05-04 07:24:31] INFO: Use the visual metrics portable script
[2023-05-04 07:24:31] INFO: Get visual metrics from the video
[2023-05-04 07:24:32] INFO: https://google.com TTFB: 252ms DOMContentLoaded: 337ms firstPaint: 317ms FCP: 317ms LCP: 340ms Load: 414ms
[2023-05-04 07:24:32] INFO: VisualMetrics: FirstVisualChange: 300ms SpeedIndex: 330ms VisualComplete85: 333ms LastVisualChange: 367ms
[2023-05-04 07:24:32] INFO: Testing url https://google.com iteration 3
[2023-05-04 07:24:41] INFO: Take after page complete check screenshot
[2023-05-04 07:24:41] INFO: Take cumulative layout shift screenshot
[2023-05-04 07:24:41] INFO: Take largest contentful paint screenshot
[2023-05-04 07:24:44] INFO: Use the visual metrics portable script
[2023-05-04 07:24:44] INFO: Get visual metrics from the video
[2023-05-04 07:24:45] INFO: https://google.com TTFB: 248ms DOMContentLoaded: 342ms firstPaint: 368ms FCP: 368ms LCP: 368ms Load: 419ms
[2023-05-04 07:24:45] INFO: VisualMetrics: FirstVisualChange: 400ms SpeedIndex: 400ms VisualComplete85: 400ms LastVisualChange: 400ms
[2023-05-04 07:24:45] INFO: https://google.com 13 requests, TTFB: 252ms (σ2.00ms 0.8%), firstPaint: 368ms (σ41.00ms 11.2%), firstVisualChange: 400ms (σ57.00ms 15.0%), FCP: 368ms (σ41.00ms 11.2%), DOMContentLoaded: 342ms (σ31.00ms 8.7%), LCP: 368ms (σ32.00ms 8.6%), CLS: 0 (σ0.00 0%), TBT: 0ms (σ0.00ms 0%), CPUBenchmark: 41ms (σ3.00ms 7.2%), Load: 419ms (σ44.00ms 9.9%), speedIndex: 400ms (σ43.00ms 11.2%), visualComplete85: 400ms (σ42.00ms 10.8%), lastVisualChange: 400ms (σ42.00ms 10.1%) (3 runs)
[2023-05-04 07:24:47] INFO: HTML stored in /sitespeed.io/output)
soulgalore commented 1 year ago

Hi @AlanCodega thanks for the detailed report!

For the Chrome user experience report, that data is collected directly from Google, so to get that dashboard to work, you need add your crux key, checkout the documentation here: https://www.sitespeed.io/documentation/sitespeed.io/crux/ That data has other structure, so that needs to be a specific dashboard.

That's good news that the slug ends up in Grafana, then there's no problem sending/getting the metrics. Maybe something is broken with dashboard? If you check a graph like https://dashboard.sitespeed.io/d/9NDMzFfMk/page-metrics-desktop in your setup, do the drop-down get any data? Do you see any metrics?

Best Peter

AlanCodega commented 1 year ago

Hey @soulgalore , Many thanks for your prompt reply. solved.