ricoberger / script_exporter

Prometheus exporter to execute scripts and collect metrics from the output or the exit status.
MIT License
354 stars 82 forks source link

used in servicemonitor #127

Closed seaurching closed 7 months ago

seaurching commented 7 months ago

how to use it in servicemonitor:

spec:
  endpoints:
  - interval: 5s
    metricRelabelings:
    - action: replace
      sourceLabels:
      - __param_target
      targetLabel: target
    params:
      output:
      - ignore
      params:
      - target
      prefix:
      - test
      script:
      - ping
      target:
      - 10.101.5.127
    path: /probe
    port: http

error logs is

ts=2024-04-12T07:53:15.467Z caller=scripts.go:92 level=error msg="Script 'ping' execution failed" cmd="/etc/script-exporter/ping.sh " stdout= stderr= env= err="exit status 2"
ts=2024-04-12T07:53:20.466Z caller=scripts.go:92 level=error msg="Script 'ping' execution failed" cmd="/etc/script-exporter/ping.sh " stdout= stderr= env= err="exit status 2"
ricoberger commented 7 months ago

Hi @seaurching, the following ServiceMonitor is working for me:

---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  namespace: kube-system
  name: script-exporter
  labels:
    app: script-exporter
spec:
  endpoints:
    - interval: 30s
      path: /probe
      port: http
      scrapeTimeout: 30s
      params:
        script:
          - ping
        params:
          - target
        target:
          - 10.244.19.246
      metricRelabelings:
        - action: replace
          replacement: ping
          targetLabel: script
      relabelings:
        - action: replace
          replacement: ping
          targetLabel: script
  namespaceSelector:
    matchNames:
      - kube-system
  selector:
    matchLabels:
      app.kubernetes.io/name: script-exporter

Please let me know if this fixes your problem.

seaurching commented 7 months ago

Hi @seaurching, the following ServiceMonitor is working for me:

---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  namespace: kube-system
  name: script-exporter
  labels:
    app: script-exporter
spec:
  endpoints:
    - interval: 30s
      path: /probe
      port: http
      scrapeTimeout: 30s
      params:
        script:
          - ping
        params:
          - target
        target:
          - 10.244.19.246
      metricRelabelings:
        - action: replace
          replacement: ping
          targetLabel: script
      relabelings:
        - action: replace
          replacement: ping
          targetLabel: script
  namespaceSelector:
    matchNames:
      - kube-system
  selector:
    matchLabels:
      app.kubernetes.io/name: script-exporter

Please let me know if this fixes your problem.

I'm sorry。There are still errors show exit status 2.

apiVersion: v1
data:
  config.yaml: |
    scripts:
      - name: ping
        script: /etc/script-exporter/ping.sh
  ping.sh: |
    #!/bin/bash
    ping -c 3 $1 > /dev/null 2>&1
    exit $?
kind: ConfigMap

---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  generation: 8
  labels:
    app.kubernetes.io/instance: script-exporter
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: scripts-exporter
    app.kubernetes.io/version: v2.18.0
    helm.sh/chart: script-exporter-0.8.1
    k8s-app: scripts-exporter
  name: scripts-exporter-ping
  namespace: default
  resourceVersion: "1901831491"
  uid: 45f06fc8-3796-4421-80dd-72e0aeca7388
spec:
  endpoints:
  - interval: 5s
    metricRelabelings:
    - action: replace
      replacement: ping
      targetLabel: script
    params:
      params:
      - target
      prefix:
      - test
      script:
      - ping
      target:
      - 10.101.5.127
    path: /probe
    port: http
    relabelings:
    - action: replace
      replacement: ping
      targetLabel: script
  selector:
    matchLabels:
      app.kubernetes.io/instance: script-exporter
      app.kubernetes.io/name: scripts-exporter

The deployment image version is 2.15.1

ricoberger commented 7 months ago

Mh if it still shows exit status 2, I think the cmd part in the log line also doesn't contain the IP address and looks like this cmd="/etc/script-exporter/ping.sh " right?

If this is the case, can you try to rename the target parameter:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  generation: 8
  labels:
    app.kubernetes.io/instance: script-exporter
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: scripts-exporter
    app.kubernetes.io/version: v2.18.0
    helm.sh/chart: script-exporter-0.8.1
    k8s-app: scripts-exporter
  name: scripts-exporter-ping
  namespace: default
  resourceVersion: "1901831491"
  uid: 45f06fc8-3796-4421-80dd-72e0aeca7388
spec:
  endpoints:
  - interval: 5s
    metricRelabelings:
    - action: replace
      replacement: ping
      targetLabel: script
    params:
      params:
      - pingtarget
      prefix:
      - test
      script:
      - ping
      pingtarget:
      - 10.101.5.127
    path: /probe
    port: http
    relabelings:
    - action: replace
      replacement: ping
      targetLabel: script
  selector:
    matchLabels:
      app.kubernetes.io/instance: script-exporter
      app.kubernetes.io/name: scripts-exporter
seaurching commented 7 months ago

@ricoberger Use this works will.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    app.kubernetes.io/instance: script-exporter
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: scripts-exporter
    app.kubernetes.io/version: v2.18.0
    helm.sh/chart: script-exporter-0.8.1
    k8s-app: scripts-exporter
  name: scripts-exporter-ping
  namespace: default
spec:
  endpoints:
  - honorLabels: true
    params:
      params:
      - target
      script:
      - ping
      target:
      - kce.internal.api.ksyun.com
      - yum.ksyun.cn
    path: /probe
    port: http
    relabelings:
    - action: replace
      regex: (.*)
      replacement: $1
      sourceLabels:
      - __meta_kubernetes_node_name
      targetLabel: node
    - action: replace
      regex: (.*)
      replacement: $1
      sourceLabels:
      - __param_target
      targetLabel: domain
    - action: replace
      regex: (.*)
      replacement: ${1}:9469
      sourceLabels:
      - __meta_kubernetes_pod_ip
      targetLabel: __address__
  namespaceSelector:
    matchNames:
    - default
  selector:
    matchLabels:
      app.kubernetes.io/instance: script-exporter
      app.kubernetes.io/name: scripts-exporter

Otherwides, the params.target add multi targets, and use relabeings to add target="domain", like bellow Capture20240419-120439

ricoberger commented 7 months ago

Hi @seaurching, thanks for sharing your solution đŸ™‚