Help with remote collection | connection timeout

[lZzozZl]

Hi all. I have my Prometheus server on Debian 12. i have installed freeipmi with apt

Im trying to get the IPMI information remote, using remote method. The ipmi_exporter agent is still on same host where is the server.

Prometheus - config

  - job_name: ipmi
    params:
      mocule: ['default']
    scrape_interval: 1m
    scrape_timeout: 30s                                                                                                                       
    metrics_path: /ipmi
    scheme: http
    file_sd_configs:
    - files:
      - /home/o-monitor/prometheus-2.50.1.linux-amd64/targets.yml
      refresh_interval: 5m
    relabel_configs:
    - source_labels: [__address__]
      separator: ;
      regex: (.*)
      target_label: __param_target
      replacement: ${1}
      action: replace
    - source_labels: [__param_target]
      separator: ;
      regex: (.*)
      target_label: instance
      replacement: ${1}
      action: replace
    - separator: ;
      regex: .*
      target_label: __address__
      replacement: ipmi-exporter.internal.example.com:9290
      action: replace

targets.yml

---
- targets:
  - 129.168.111.111
  labels:
    job: ipmi_exporter

ipmi-exporter config

modules:
  default:
    # These settings are used if no module is specified, the
    # specified module doesn't exist, or of course if
    # module=default is specified.
    user: "IPMI-USER"
    pass: "IPMI-PASS"
    # The below settings correspond to driver-type, privilege-level, and
    # session-timeout respectively, see `man 5 freeipmi.conf` (and e.g.
    # `man 8 ipmi-sensors` for a list of driver types).
    #driver: "LAN"
    driver: "LAN_2_0"
    #privilege: "user"
    privilege: "admin"
    # The session timeout is in milliseconds. Note that a scrape can take up
    # to (session-timeout * #-of-collectors) milliseconds, so set the scrape
    # timeout in Prometheus accordingly.
    # Must be larger than the retransmission timeout, which defaults to 1000.
    timeout: 10000
    # Available collectors are bmc, bmc-watchdog, ipmi, chassis, dcmi, sel,
    # and sm-lan-mode
    # If _not_ specified, bmc, ipmi, chassis, and dcmi are used
    collectors:
    - ipmi
    - bmc
    - chassis
    collector_cmd:
      ipmi: sudo
      bmc: sudo
      chassis: sudo
    custom_args:
      ipmi:
      - "ipmimonitoring"
      bmc:
      - "bmc-info"
      chassis:
      - "ipmi-chassis"

I got this problem from ipmi-exporter agent

ts=2024-03-13T11:09:05.810Z caller=collector_ipmi.go:151 level=error msg="Failed to collect sensor data" target=1.2.3.4 error="error running sudo: exit status 1: /usr/sbin/ipmi-sensors: connection timeout\n" ts=2024-03-13T11:09:17.828Z caller=collector_bmc.go:53 level=error msg="Failed to collect BMC data" target=1.2.3.4 error="error running sudo: exit status 1: bmc-info: connection timeout\n" ts=2024-03-13T11:09:29.842Z caller=collector_chassis.go:65 level=error msg="Failed to collect chassis data" target=1.2.3.4 error="error running sudo: exit status 1: ipmi-chassis: connection timeout\n"

[bitfehler]

Your logs say target=1.2.3.4 - did you redact the actual IP? I am asking, because you did not redact it your targets.yaml. So maybe you are trying to scrape the wrong address?

Note that either way, this is a configuration issue on your side, potentially at a lower level even then the exporter. The only way I'd be able to help you is if you can provide a command that successfully runs e.g. bmc-info on the server running the exporter against one of your desired targets.