ricoberger / script_exporter

Prometheus exporter to execute scripts and collect metrics from the output or the exit status.
MIT License
354 stars 82 forks source link

Document what gets parsed out of a script's stdout and sent with the probe #167

Open dvtsten opened 1 week ago

dvtsten commented 1 week ago

Looking at the source: https://github.com/ricoberger/script_exporter/blob/39b1980d7ae78d75b39fc3f454f189e0d11e911d/pkg/exporter/metrics.go#L108

it seems like I have to pass a regexp match in order for output to be sent with my probe response.

When I use this script:

#!/usr/bin/env bash

netstat -tn | awk '/ESTABLISHED/ {print $4,$5}' | while read conn; do
  conn=($conn)
  echo "connection_established{destination="${conn[1]}",source="${conn[0]}"}"
done

I see this from the debug log for script_exporter:

ts=2024-11-12T16:23:18.870Z caller=scripts.go:108 level=debug msg="Script 'connection_list' execution succeed" cmd=./check_connections.sh stdout="connection_established{destination=3.22.86.39:443,source=10.10.52.36:32852}\nconnection_established{destination=35.162.224.228:443,source=10.10.52.36:44005}\nconnection_established{destination=10.35.244.19:65387,source=10.10.52.36:22}\nconnection_established{destination=10.10.113.83:389,source=10.10.52.36:46220}\nconnection_established{destination=10.35.244.19:54421,source=10.10.52.36:9469}\n" stderr= env=

but I don't see any of that in the probe output:

$ curl -v http://localhost:9469/probe?script=connection_list
*   Trying 127.0.0.1:9469...
* Connected to localhost (127.0.0.1) port 9469 (#0)
> GET /probe?script=connection_list HTTP/1.1
> Host: localhost:9469
> User-Agent: curl/7.76.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Date: Tue, 12 Nov 2024 16:25:12 GMT
< Content-Length: 734
<
# HELP script_success Script exit status (0 = error, 1 = success).
# TYPE script_success gauge
script_success{script="connection_list"} 1
# HELP script_duration_seconds Script execution time, in seconds.
# TYPE script_duration_seconds gauge
script_duration_seconds{script="connection_list"} 0.003175
# HELP script_exit_code The exit code of the script.
# TYPE script_exit_code gauge
script_exit_code{script="connection_list"} 0
# HELP script_use_cache Script use cache (0 = no, 1 = yes).
# TYPE script_use_cache gauge
script_use_cache{script="connection_list"} 0
# HELP script_use_expired_cache Script re-use expired cache (0 = no, 1 = yes).
# TYPE script_use_expired_cache gauge
script_use_expired_cache{script="connection_list"} 0

* Connection #0 to host localhost left intact

In fussing with this, I changed my script to do this:

echo "connection_established{destination="${conn[1]}",source="${conn[0]}"} 1"

adding a " 1" to the end of each line, and after doing that, I got the output I was expecting to see from the probe.

Can you document how that regex filters output and what is expected to be sent?