grafana / synthetic-monitoring-agent

Synthetic Monitoring Agent
https://grafana.com/docs/grafana-cloud/how-do-i/synthetic-monitoring/
Apache License 2.0
164 stars 24 forks source link

Add support for k6 builtin metrics #942

Open d0ugal opened 1 week ago

d0ugal commented 1 week ago

While working on #832 I noticed that we don't have good support for all the built-in k6 metrics.

When using this command

./k6 run script.js --out=sm=sm.metrics --out=json=metrics.json --quiet

The following grpc check completes successfully (with this fix https://github.com/grafana/xk6-sm/pull/3)

import { Client, StatusOK } from 'k6/net/grpc';
import { check, sleep } from 'k6';

const client = new Client({});

export default () => {
  client.connect('grpcbin.test.k6.io:9001', {
    reflect: true
  });

  const data = { greeting: 'Bert' };
  const response = client.invoke('hello.HelloService/SayHello', data);

  console.log(response.status)

  check(response, {
    'status is OK': (r) => r && r.status === StatusOK,
  });

  console.log(JSON.stringify(response.message));

  client.close();
};

However, if we compare the SM metrics with the standard k6 metrics we can see a problem.

probe_vus_max{} 1
probe_checks_total{scenario="default",result="pass"} 1
probe_data_sent_bytes{scenario="default"} 3115
probe_data_received_bytes{scenario="default"} 38205
probe_iteration_duration_seconds{scenario="default"} 1.789952416
# HELP probe_script_duration_seconds Returns how long the script took to complete in seconds
probe_script_duration_seconds{} 1.794847667
probe_grpc_req_duration{status="12",scenario="default"} 91.974083
probe_vus{} 1
probe_http_got_expected_response{url="grpcbin.test.k6.io:9001/hello.HelloService/SayHello",method="SayHello",scenario="default",name="/hello.HelloService/SayHello"} 1
probe_http_error_code{url="grpcbin.test.k6.io:9001/hello.HelloService/SayHello",method="SayHello",scenario="default",name="/hello.HelloService/SayHello"} 0
probe_http_info{service="hello.HelloService",url="grpcbin.test.k6.io:9001/hello.HelloService/SayHello",method="SayHello",scenario="default",name="/hello.HelloService/SayHello"} 1
probe_http_requests_total{url="grpcbin.test.k6.io:9001/hello.HelloService/SayHello",method="SayHello",scenario="default",name="/hello.HelloService/SayHello"} 0
probe_http_requests_failed_total{url="grpcbin.test.k6.io:9001/hello.HelloService/SayHello",method="SayHello",scenario="default",name="/hello.HelloService/SayHello"} 0
probe_http_ssl{url="grpcbin.test.k6.io:9001/hello.HelloService/SayHello",method="SayHello",scenario="default",name="/hello.HelloService/SayHello"} 0
{"type":"Metric","data":{"name":"grpc_req_duration","type":"trend","contains":"time","thresholds":[],"submetrics":null},"metric":"grpc_req_duration"}
{"metric":"grpc_req_duration","type":"Point","data":{"time":"2024-10-15T13:21:41.974465+01:00","value":91.974083,"tags":{"group":"","scenario":"default","status":"12"}}}
{"type":"Metric","data":{"name":"vus","type":"gauge","contains":"default","thresholds":[],"submetrics":null},"metric":"vus"}
{"metric":"vus","type":"Point","data":{"time":"2024-10-15T13:21:42.538538+01:00","value":1,"tags":{}}}
{"type":"Metric","data":{"name":"vus_max","type":"gauge","contains":"default","thresholds":[],"submetrics":null},"metric":"vus_max"}
{"metric":"vus_max","type":"Point","data":{"time":"2024-10-15T13:21:42.538538+01:00","value":1,"tags":{}}}
{"metric":"grpc_req_duration","type":"Point","data":{"time":"2024-10-15T13:21:43.32794+01:00","value":91.425041,"tags":{"group":"","method":"SayHello","name":"/hello.HelloService/SayHello","scenario":"default","service":"hello.HelloService","status":"0","url":"grpcbin.test.k6.io:9001/hello.HelloService/SayHello"}}}
{"type":"Metric","data":{"name":"checks","type":"rate","contains":"default","thresholds":[],"submetrics":null},"metric":"checks"}
{"metric":"checks","type":"Point","data":{"time":"2024-10-15T13:21:43.328347+01:00","value":1,"tags":{"check":"status is OK","group":"","scenario":"default"}}}
{"type":"Metric","data":{"name":"data_sent","type":"counter","contains":"data","thresholds":[],"submetrics":null},"metric":"data_sent"}
{"metric":"data_sent","type":"Point","data":{"time":"2024-10-15T13:21:43.328808+01:00","value":3115,"tags":{"group":"","scenario":"default"}}}
{"type":"Metric","data":{"name":"data_received","type":"counter","contains":"data","thresholds":[],"submetrics":null},"metric":"data_received"}
{"metric":"data_received","type":"Point","data":{"time":"2024-10-15T13:21:43.328808+01:00","value":38205,"tags":{"group":"","scenario":"default"}}}
{"type":"Metric","data":{"name":"iteration_duration","type":"trend","contains":"time","thresholds":[],"submetrics":null},"metric":"iteration_duration"}
{"metric":"iteration_duration","type":"Point","data":{"time":"2024-10-15T13:21:43.328808+01:00","value":1789.952416,"tags":{"group":"","scenario":"default"}}}
{"type":"Metric","data":{"name":"iterations","type":"counter","contains":"default","thresholds":[],"submetrics":null},"metric":"iterations"}
{"metric":"iterations","type":"Point","data":{"time":"2024-10-15T13:21:43.328808+01:00","value":1,"tags":{"group":"","scenario":"default"}}}

The xk6-sm output contains http metrics but k6 doesn't create any. So we are inferring some http metrics from the grpc metrics but missing most of the grpc metrics still.

peterschretlen commented 1 day ago

For any protocols we want to support: grpc, websocket. We should make sure the metrics are coming through as expected.