Empty response from server

[vshmakov-vcl]

Hi. I'm using idrac_exporter with 3 Dell server, everything ok. But recently I've get another one server (all same R660), but with newer software. And when I try to get metrics via curl, I've get:

$ curl http://localhost:9348/metrics?target=192.168.1.10   
curl: (52) Empty reply from server

iDRAC's Firmware versions is same, new server get me response via Redfish's API.

[mrlhansen]

Hi @vshmakov-vcl

There is probably an error in the log for the exporter. What does it say?

[vshmakov-vcl]

I've found this record in journalctl:

May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: 2024/05/30 17:10:58 http: panic serving 127.0.0.1:33266: runtime error: invalid memory address or nil pointer dereference
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: goroutine 76108 [running]:
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.(*conn).serve.func1()
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:1898 +0xbe
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: panic({0x7dba60?, 0xb761f0?})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/runtime/panic.go:770 +0x132
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: github.com/mrlhansen/idrac_exporter/internal/config.(*RootConfig).GetHostCfg(0xb8b8a0, {0xc00002c764, 0xf})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /home/monitor/go/pkg/mod/github.com/mrlhansen/idrac_exporter@v1.2.1/internal/config/config.go:45 +0x10f
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: github.com/mrlhansen/idrac_exporter/internal/collector.GetCollector({0xc00002c764, 0xf})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /home/monitor/go/pkg/mod/github.com/mrlhansen/idrac_exporter@v1.2.1/internal/collector/collector.go:438 +0x1b4
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: main.MetricsHandler({0x8fded8, 0xc0001a61c0}, 0xc0000c2900)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /home/monitor/go/pkg/mod/github.com/mrlhansen/idrac_exporter@v1.2.1/cmd/idrac_exporter/handler.go:54 +0x131
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.HandlerFunc.ServeHTTP(0xb8be90?, {0x8fded8?, 0xc0001a61c0?}, 0x65a73a?)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:2166 +0x29
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.(*ServeMux).ServeHTTP(0x469859?, {0x8fded8, 0xc0001a61c0}, 0xc0000c2900)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:2683 +0x1ad
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.serverHandler.ServeHTTP({0xc0003c40f0?}, {0x8fded8?, 0xc0001a61c0?}, 0x6?)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:3137 +0x8e
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.(*conn).serve(0xc0001b4090, {0x8fe770, 0xc000180c30})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:2039 +0x5e8
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: created by net/http.(*Server).Serve in goroutine 1
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:3285 +0x4b4
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: 2024/05/30 17:10:58 http: panic serving 127.0.0.1:33434: runtime error: invalid memory address or nil pointer dereference
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: goroutine 76149 [running]:
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.(*conn).serve.func1()
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:1898 +0xbe
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: panic({0x7dba60?, 0xb761f0?})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/runtime/panic.go:770 +0x132
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: github.com/mrlhansen/idrac_exporter/internal/config.(*RootConfig).GetHostCfg(0xb8b8a0, {0xc0000a2854, 0xf})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /home/monitor/go/pkg/mod/github.com/mrlhansen/idrac_exporter@v1.2.1/internal/config/config.go:45 +0x10f
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: github.com/mrlhansen/idrac_exporter/internal/collector.GetCollector({0xc0000a2854, 0xf})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /home/monitor/go/pkg/mod/github.com/mrlhansen/idrac_exporter@v1.2.1/internal/collector/collector.go:438 +0x1b4
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: main.MetricsHandler({0x8fded8, 0xc00038e2a0}, 0xc00012b680)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /home/monitor/go/pkg/mod/github.com/mrlhansen/idrac_exporter@v1.2.1/cmd/idrac_exporter/handler.go:54 +0x131
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.HandlerFunc.ServeHTTP(0xb8be90?, {0x8fded8?, 0xc00038e2a0?}, 0x65a73a?)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:2166 +0x29
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.(*ServeMux).ServeHTTP(0x469859?, {0x8fded8, 0xc00038e2a0}, 0xc00012b680)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:2683 +0x1ad
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.serverHandler.ServeHTTP({0xc000246ff0?}, {0x8fded8?, 0xc00038e2a0?}, 0x6?)
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:3137 +0x8e
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: net/http.(*conn).serve(0xc000216870, {0x8fe770, 0xc000180c30})
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:2039 +0x5e8
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]: created by net/http.(*Server).Serve in goroutine 1
May 30 17:10:58 mon-sys prometheus-idrac-exporter[498]:         /usr/local/go/src/net/http/server.go:3285 +0x4b4

[mrlhansen]

I would guess that you do not have a "default" host in your config file and you have not added the login information for the new IP address. I should implement better error handling for this scenario.

[vshmakov-vcl]

Yes, I have no "default" section, but I've define creds for this host

[mrlhansen]

Did you restart the exporter after adding the information? It is failing because it does not find the host information.

[vshmakov-vcl]

Yes, I've restart Prometheus a few times, same effect

[mrlhansen]

You need to restart the idrac_exporter, not Prometheus. I added a fix, such that it will print an error instead of crashing (will be included in the next release).

[vshmakov-vcl]

Now work's, my bad. Thank you!