cannot match salt job with salt response (wrong parsing of salt job?)

[Grayfoox]

hi,

not sure if its error on my side or not but exporter shows "job" as salt01, and not as job_ID. which makes hard to match new job and return for failed jobs.

currently running on salt 3005.1 0.8.0 version of salt-exporter

thanks

[kpetremann]

Hello @Grayfoox,

The job label is not exposed by the exporter. This is a default label Prometheus is adding when scraping the metrics. It comes from your Prometheus configuration, which should look like:

scrape_configs:
  - job_name: salt1    #  <<<<<<<<<<<<< this is the value that Prometheus expose in "job" label
    static_configs:
      - targets: ["127.0.0.1:2112"]

The Salt Job ID is not exposed because it would increase way too much the cardinality (number of metrics).

See the exposed labels (there are examples at the end of the page): https://kpetremann.github.io/salt-exporter/salt-exporter/metrics/

[Grayfoox]

thanks for info.

is there different way to match job and ret code?

I guess trough minion name and timedate.

[kpetremann]

you can use the salt_responses_total or salt_scheduled_job_return_total, but it depends on how often you have jobs VS scraping interval.

Also, if you want to watch only one state or function, you can use salt_function_status and configure both filters to get what you want per minion:

  salt_function_status:
    enabled: true
    filters:
      functions:
        - "<the function you want>"
      states:
        - "<the state you want if needed>"

It will give you a metric per minion, function and state: salt_function_status{function="state.highstate",minion="node1",state="highstate"} 1

[kpetremann]

I am closing this issue. Do not hesistate to re-open this one or another if needed.