Failure instantiating the UPS

[tsull360]

Hello, New to Prometheus/Grafana in general, this may just be learning pains.

I've got 2 separate Raspberry Pi's connected to 2 different UPS, each running the exporter. My scraper is configured as follows:

• job_name: "nut-NATUPS01" metrics_path: '/ups_metrics' static_configs:

  • targets: ["192.168.50.30:9199"] labels: ups: "NATUPS01" params: ups: [ "NATUPS01" ]

• job_name: "nut-NATUPS02" metrics_path: '/ups_metrics' static_configs:

  • targets: ["192.168.50.31:9199"] labels: ups: "NATUPS02" params: ups: [ "NATUPS02" ]

However no data seems to be making it into prometheus, and I see the following output from my exporters: ts=2022-12-27T17:34:15.303Z caller=nut_collector.go:95 level=error msg="Failure instantiating the UPS" name=NATUPS02 err="error reading response: EOF" 2022/12/27 10:34:44 http: panic serving 192.168.10.36:52071: duplicate metrics collector registration attempted goroutine 65 [running]: net/http.(conn).serve.func1() net/http/server.go:1825 +0xec panic({0x477ab0, 0x1c0c400}) runtime/panic.go:844 +0x2ac github.com/prometheus/client_golang/prometheus.(Registry).MustRegister(...) github.com/prometheus/client_golang@v1.11.0/prometheus/registry.go:403 main.(metricsHandler).ServeHTTP(0x1d08f90, {0x5527ac, 0x1faa000}, 0x1c68c00) github.com/DRuggeri/nut_exporter/nut_exporter.go:146 +0x89c net/http.(ServeMux).ServeHTTP(0x7eb818, {0x5527ac, 0x1faa000}, 0x1c68c00) net/http/server.go:2462 +0x164 net/http.serverHandler.ServeHTTP({0x1d2cd80}, {0x5527ac, 0x1faa000}, 0x1c68c00) net/http/server.go:2916 +0x408 net/http.(conn).serve(0x1cda600, {0x552ab0, 0x1d26a08}) net/http/server.go:1966 +0x654 created by net/http.(Server).Serve net/http/server.go:3071 +0x4e0

Any pointers into what I'm doing wrong?

[DRuggeri]

Hi, @tsull360 - this is interesting... I haven't tried to run the exporter with two UPS devices so you may have uncovered a potential bug. I see two issues:

• error reading response: EOF coming from the go.nut client. This indicates some kind of issue between nut_exporter and the NUT server.
• duplicate metrics collector registration attempted looks like a potential issue with the collector and how it dynamically registers. This might be caused by concurrent scrapes or just fallout from the error in the NUT client

I think both issues could be examined a bit better if you can share some debug logs. If the first issue is related to concurrency, I'd expect interleaved logs or maybe some kind of breadcrumb trail. The second issue, I'm a little unsure about. I would think the EOF could indicate that the NUT server is rejecting the client (is authentication configured?), the client is not connecting, the NUT server is hanging, or some other issue. Confirming that the NUT servers are functional with a regular old NUT client from command line on both Pis could maybe expose any such issues?

[DRuggeri]

With a quick test, I was able to duplicate both problems by providing a UPS name that doesn't exist in the NUT server. What seems to happen:

• The request to the NUT server for the UPS info is either never responded to, or the go.nut client doesn't handle the error... thus it sits and spins indefinitely
• A follow up request to the exporter, while the first one is hung, triggers the MustRegister issue

I have a few ideas of how I can work on this in the code, but it'd be helpful if you can identify the cause of the issue with the NUT server. I'm guessing either a connection hang or an invalid name according to the NUT server. That info would help guide next steps.

[DRuggeri]

I just released a fix for the concurrency issue. If the problem in your configuration is related to an invalid UPS name, that will also show up as an error in the log.

[tsull360]

Thank you! The increased information has helped give me a pointer to my misconfiguration. Just need to figure out where now :)

pi@natups01:~ $ ./nut_exporter-v2.5.1-linux-arm --log.level=debug ts=2022-12-29T17:52:25.147Z caller=nut_exporter.go:256 level=info msg="Starting nut_exporter" version=2.5.1 ts=2022-12-29T17:52:25.148Z caller=nut_exporter.go:284 level=info msg="Listening on" address=:9199 ts=2022-12-29T17:52:29.946Z caller=nut_exporter.go:145 level=info msg="Creating new registry, handler, and collector for UPS named NATUPS01" ts=2022-12-29T17:52:29.948Z caller=nut_collector.go:301 level=debug msg="Connecting to server and verifying NATUPS01 is a valid UPS name" server=127.0.0.1 ts=2022-12-29T17:52:30.168Z caller=nut_collector.go:325 level=debug msg="UPS name detection" name=lab ts=2022-12-29T17:52:30.168Z caller=nut_collector.go:331 level=debug msg="Validity result for UPS named NATUPS01" valid=false ts=2022-12-29T17:52:30.169Z caller=nut_exporter.go:154 level=error msg="Internal server error" err="NATUPS01 UPS is not a valid name in the NUT server" ts=2022-12-29T17:52:44.944Z caller=nut_exporter.go:145 level=info msg="Creating new registry, handler, and collector for UPS named NATUPS01"

[DRuggeri]

Cool - believe it or not, it's all right there for you.

This line...

ts=2022-12-29T17:52:30.168Z caller=nut_collector.go:325 level=debug msg="UPS name detection" name=lab

... tells us that the NUT server reported that there is a UPS named "lab", so this should be the value you use for the ups parameter. Alternatively, since there is only one UPS attached to this NUT server, you could skip setting the ups query string parameter altogether and the exporter will identify this one UPS and scrape it.

[DRuggeri]

Hopefully that gets you squared away, @tsull360 - feel free to reopen if not