prometheus / snmp_exporter

SNMP Exporter for Prometheus
Apache License 2.0
1.66k stars 617 forks source link

Add support for entPhySensorTable values #1066

Open lukeyeager opened 9 months ago

lukeyeager commented 9 months ago

Host operating system: output of uname -a

$ uname -srmo
Linux 5.15.0-78-generic x86_64 GNU/Linux

snmp_exporter version: output of snmp_exporter -version

ts=2023-12-07T20:57:47.617Z caller=main.go:194 level=info msg="Starting snmp_exporter" version="(version=0.24.1, branch=HEAD, revision=f8826bcfadf9d93df9516c660fdc1d503f4e0b95)" concurrency=1

What device/snmpwalk OID are you using?

An Infiniband switch, using ENTITY-SENSOR-MIB (RFC 3433).

If this is a new device, please link to the MIB(s).

This is a standard MIB. You can download it using the Dockerfile below.

What did you do that produced an error?

No errors, but here is my setup.

generator.yml

auths:
  public_v2:
    community: public
    security_level: noAuthNoPriv
    auth_protocol: MD5
    priv_protocol: DES
    version: 2
modules:
  entity-sensors:
    walk:
      - entPhysicalIndex
      - entPhySensorScale
      - entPhySensorPrecision
      - entPhySensorValue
    lookups:
      - source_indexes: [entPhysicalIndex]
        lookup: entPhySensorScale
      - source_indexes: [entPhysicalIndex]
        lookup: entPhySensorPrecision
    overrides:
      entPhysicalIndex: {ignore: true}
      entPhySensorScale: {ignore: true}
      entPhySensorPrecision: {ignore: true}

Dockerfile

FROM golang:1.19-bullseye
RUN sed -i 's/main$/main non-free/' /etc/apt/sources.list
RUN apt-get update \
    && apt-get install -y --no-install-recommends build-essential libsnmp-dev snmp-mibs-downloader unzip \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /opt
RUN git clone --depth=1 --branch=v0.24.1 "https://github.com/prometheus/snmp_exporter.git" .
RUN make build && make -C generator generator
COPY generator.yml generator/
RUN cd generator && ./generator --fail-on-parse-errors generate && mv snmp.yml ../
ENTRYPOINT [ "./snmp_exporter" ]
$ docker build -t tmp . && docker run --rm -it -p 9116:9116 tmp
$ curl -fsS 'localhost:9116/snmp?target=myswitch&module=entity-sensors' | head -n3
# HELP entPhySensorValue The most recent measurement obtained by the agent for this sensor - 1.3.6.1.2.1.99.1.1.1.4
# TYPE entPhySensorValue gauge
entPhySensorValue{entPhySensorPrecision="0",entPhySensorScale="8",entPhysicalIndex="601240040"} 700

What did you expect to see?

The actual value for this sensor is 0.7. The user must combine scale, precision, and value to calculate this final value. In our custom exporter, we use the following snippet to do the conversion:

func phySensorConvertValue(value int64, scale int64, precision int64) (float64, error) {
    if scale < 1 || scale > 17 {
        return 0, fmt.Errorf("entPhySensorScale value %d is outside range [1, 17]", scale)
    }
    if precision < -8 || precision > 9 {
        return 0, fmt.Errorf("entPhySensorPrecision value %d is outside range [-8, 9]", precision)
    }
    exponent := 3*(scale-9) - precision
    return float64(value) * math.Pow10(int(exponent)), nil
}

What did you see instead?

Currently, snmp-exporter can only report the individual values in separate series. Since this is a standard MIB, I was wondering if you might consider adding new functionality to modules.NAME.overrides (possibly a new "type"?) to handle this automatically.

SuperQ commented 9 months ago

We just recently added a scale configuration option to the code. In theory, we could have an override that looks up the scale factor from another OID.

lukeyeager commented 9 months ago

My workaround is not too bad. The downsides are that it moves some logic into prometheus' recording rules and that it adds a little too much data to the database that I don't really want.

generator.yml

modules:
  entity-physical-sensors:
    walk:
      - entPhysicalIndex
      - entPhysicalDescr
      - entPhySensorScale
      - entPhySensorPrecision
      - entPhySensorValue
    lookups:
      - source_indexes: [entPhysicalIndex]
        lookup: entPhysicalDescr
    overrides:
      entPhysicalIndex: {ignore: true}
      entPhysicalDescr: {ignore: true}

prometheus/rules-recording.yml

groups:
  - name: snmp-exporter
    rules:
      - record: entPhySensorValueScaled
        expr: entPhySensorValue * 10 ^ (3 * (entPhySensorScale - 9) - entPhySensorPrecision)

@SuperQ yes, I noticed that new static field. But I would need something to handle the formula valueScaled = value * 10 ^ (3 * (scale - 9) - precision). You do a static scale, but I would need to be able to compose add, subtract, multiply, and pow. Which is a lot of new functionality. That, or add a special case for entPhySensorTable.

I'm content with a "no, that's out of scope." But I thought I'd ask just in case.

SuperQ commented 9 months ago

I think it's something we could do in code, but it would need to be generic enough to work with other MIBs.

I don't have time to implement this, but PRs would be welcome.