grafana / synthetic-monitoring-agent

Synthetic Monitoring Agent
https://grafana.com/docs/grafana-cloud/how-do-i/synthetic-monitoring/
Apache License 2.0
157 stars 20 forks source link

k6runner: use check context for http request #715

Closed roobre closed 1 month ago

roobre commented 1 month ago

This PR instructs the HTTP-backed k6 runner (Grafana Cloud) to use the context carrying the script timeout as the request context for HTTP requests made to that k6 runner. An extra second of grace time is added to account for network latency is added to the check timeout.

This fixes a potential deadlock where the agent would wait for the HTTP request to complete forever, even longer than the maximum time the script is allowed to run. This does not occur frequently, but could happen on certain scenarios such as when GC runners are abruptly terminated.

Note that this is a stopgap measure to prevent deadlocks in the agent. Ideally, we should have a separate settings for the time we wait for the runner to respond, and the time that the runner allows k6 to wait for. See details about the long-term proposal here: https://docs.google.com/document/d/1kq5xC4izHyJ9WOonS308J8sE3uDG48c6gj4ZPHsWIx4/edit.