search

Luzilla / dnsbl_exporter

Prometheus compatible exporter to query DNSBLs/RBLs.
https://www.luzilla-capital.com/
Other
30 stars 8 forks source link

Exporter suddenly stop working. #159

Closed kozicpetar closed 5 months ago

kozicpetar commented 1 year ago

I have installed latest release from git.

I set exporter to work as systemd service. This is systemd file:

[Unit]
Description=RBLExporter

[Service]
TimeoutStartSec=0
User=rbl_exporter
ExecStart=/usr/bin/dnsbl-exporter --web.listen-address=0.0.0.0:9211 --config.rbls "/etc/rbl-exporter/rbl_list.ini" --config.targets "/etc/rbl-exporter/rbl_targets.ini"

[Install]
WantedBy=multi-user.target

OS is: VERSION="22.04.2 LTS (Jammy Jellyfish)"

Service working some time, sometimes several hours, sometimes days or two and suddenly stop working with error:

<autogenerated>:1 +0x29 fp=0xc0001b0f78 sp=0xc0001b0f48 pc=0x5d5c49
rbl_exporter.service: Consumed 46.831s CPU time.
net/http.(*connReader).backgroundRead(0xc0002787e0)
/opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:674 +0x3f fp=0xc0001b0fc8 sp=0xc0001b0f78 pc=0x6a821f
net/http.(*connReader).startBackgroundRead.func2()
/opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:670 +0x26 fp=0xc0001b0fe0 sp=0xc0001b0fc8 pc=0x6a8146
runtime.goexit()
/opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0001b0fe8 sp=0xc0001b0fe0 pc=0x46a341
created by net/http.(*connReader).startBackgroundRead
/opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:670 +0xca
till commented 1 year ago

I only glanced over it.

Can you possibly paste the entire journal? Or share your targets so I can try to reproduce?

I'll look again when I am in front of my computer.

till commented 1 year ago

@kozicpetar I looked again, I am assuming there is a lot more prior to this? Please paste the entire log and please format it with backticks for readability if you do. :)

kozicpetar commented 1 year ago

Hi @till sorry I can't get you my targets. I have about 10 targets which I monitoring. This is whole journal log when service suddenly stop working.

May 29 16:13:45  systemd[1]: rbl_exporter.service: Current command vanished from the unit file, execution of the command list won't be resumed.
May 29 16:13:47  systemd[1]: Stopping RBLExporter...
May 29 16:13:47  systemd[1]: rbl_exporter.service: Deactivated successfully.
May 29 16:13:47  systemd[1]: Stopped RBLExporter.
May 29 16:13:47  systemd[1]: rbl_exporter.service: Consumed 6.476s CPU time.
May 29 16:13:47  systemd[1]: Started RBLExporter.
May 29 16:13:47  dnsbl-exporter[9388]: VERSION: 0.6.0
May 29 16:13:47  dnsbl-exporter[9388]: time=2023-05-29T16:13:47.779+02:00 level=INFO msg="Starting on: 0.0.0.0:9211"
May 29 16:14:00  dnsbl-exporter[9388]: time=2023-05-29T16:14:00.253+02:00 level=ERROR msg="" area=metrics target=mail.mymailserver
May 29 16:14:00  dnsbl-exporter[9388]: time=2023-05-29T16:14:00.253+02:00 level=ERROR msg="" area=metrics target=mail.mymailserver
May 29 16:14:00  dnsbl-exporter[9388]: time=2023-05-29T16:14:00.253+02:00 level=ERROR msg="" area=metrics target=mail.mymailserver
May 29 16:28:15  dnsbl-exporter[9388]: runtime: bad pointer in frame github.com/Luzilla/dnsbl_exporter/collector.(*RblCollector).Collect at 0xc0004bdec0: 0xc
May 29 16:28:15  dnsbl-exporter[9388]: fatal error: invalid pointer found on stack
May 29 16:28:15  dnsbl-exporter[9388]: runtime stack:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.throw({0x9af0b2?, 0xd90ca0?})
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/panic.go:1047 +0x5d fp=0xc0004b1810 sp=0xc0004b17e0 pc=0x437bbd
May 29 16:28:15  dnsbl-exporter[9388]: runtime.adjustpointers(0xc0004b1c08?, 0x0?, 0xc500d7?, {0xd70db8?, 0xd90ca0?})
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/stack.go:632 +0x1d0 fp=0xc0004b1878 sp=0xc0004b1810 pc=0x450130
May 29 16:28:15  dnsbl-exporter[9388]: runtime.adjustframe(0xc0004b1c08, 0xc0004b1ce8)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/stack.go:674 +0xc9 fp=0xc0004b1908 sp=0xc0004b1878 pc=0x450249
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gentraceback(0x851b09?, 0x0?, 0x0?, 0xc0002ed6c0, 0x0, 0x0, 0x7fffffff, 0x9e1390, 0x0?, 0x0)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/traceback.go:332 +0xbed fp=0xc0004b1c60 sp=0xc0004b1908 pc=0x45b66d
May 29 16:28:15  dnsbl-exporter[9388]: runtime.copystack(0xc0002ed6c0, 0x800000002?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/stack.go:934 +0x2f5 fp=0xc0004b1e18 sp=0xc0004b1c60 pc=0x450a75
May 29 16:28:15  dnsbl-exporter[9388]: runtime.newstack()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/stack.go:1114 +0x485 fp=0xc0004b1fc8 sp=0xc0004b1e18 pc=0x450fe5
May 29 16:28:15  dnsbl-exporter[9388]: runtime.morestack()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:574 +0x8b fp=0xc0004b1fd0 sp=0xc0004b1fc8 pc=0x4683ab
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 26326 [copystack]:
May 29 16:28:15  dnsbl-exporter[9388]: sync.(*poolChain).popTail(0xc000004410?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/sync/poolqueue.go:271 +0xbd fp=0xc0004bd3a0 sp=0xc0004bd398 pc=0x48545d
May 29 16:28:15  dnsbl-exporter[9388]: sync.(*Pool).getSlow(0xdbf160, 0x1)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/sync/pool.go:163 +0x87 fp=0xc0004bd3d8 sp=0xc0004bd3a0 pc=0x484747
May 29 16:28:15  dnsbl-exporter[9388]: sync.(*Pool).Get(0xdbf160)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/sync/pool.go:140 +0x79 fp=0xc0004bd410 sp=0xc0004bd3d8 pc=0x484659
May 29 16:28:15  dnsbl-exporter[9388]: golang.org/x/exp/slog/internal/buffer.New(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/golang.org/x/exp@v0.0.0-20230425010034-47ecfdc1ba53/slog/internal/buffer/buffer.go:24
May 29 16:28:15  dnsbl-exporter[9388]: golang.org/x/exp/slog.(*commonHandler).handle(_, {{0xc1154b7ff5b93520, 0xca203fc5b8, 0xdd1980}, {0x0, 0x0}, 0x8, 0x851adb, {{{0x0, 0x0}, ...}, ...}, ...})
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/golang.org/x/exp@v0.0.0-20230425010034-47ecfdc1ba53/slog/handler.go:257 +0x57 fp=0xc0004bd7d8 sp=0xc0004bd410 pc=0x6dcbb7
May 29 16:28:15  dnsbl-exporter[9388]: golang.org/x/exp/slog.(*TextHandler).Handle(_, {_, _}, {{0xc1154b7ff5b93520, 0xca203fc5b8, 0xdd1980}, {0x0, 0x0}, 0x8, 0x851adb, ...})
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/golang.org/x/exp@v0.0.0-20230425010034-47ecfdc1ba53/slog/text_handler.go:94 +0x58 fp=0xc0004bd910 sp=0xc0004bd7d8 pc=0x6e5858
May 29 16:28:15  dnsbl-exporter[9388]: golang.org/x/exp/slog.(*Logger).log(0xc000487520, {0x0, 0x0}, 0x8, {0x0, 0x0}, {0x0, 0x0, 0x0})
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/golang.org/x/exp@v0.0.0-20230425010034-47ecfdc1ba53/slog/logger.go:232 +0x1fc fp=0xc0004bdca8 sp=0xc0004bd910 pc=0x6e3efc
May 29 16:28:15  dnsbl-exporter[9388]: golang.org/x/exp/slog.(*Logger).Error(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/golang.org/x/exp@v0.0.0-20230425010034-47ecfdc1ba53/slog/logger.go:205
May 29 16:28:15  dnsbl-exporter[9388]: github.com/Luzilla/dnsbl_exporter/collector.(*RblCollector).Collect(0xc000024780, 0xc00011af60?)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/work/dnsbl_exporter/dnsbl_exporter/collector/collector.go:133 +0x9e9 fp=0xc0004bdf30 sp=0xc0004bdca8 pc=0x851b09
May 29 16:28:15  dnsbl-exporter[9388]: github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/registry.go:455 +0x10d fp=0xc0004bdfe0 sp=0xc0004bdf30 pc=0x7d6e4d
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0004bdfe8 sp=0xc0004bdfe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/registry.go:547 +0xc09
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 1 [IO wait, 14 minutes]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000349628 sp=0xc000349608 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.netpollblock(0x7f4978617528?, 0x40482f?, 0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/netpoll.go:527 +0xf7 fp=0xc000349660 sp=0xc000349628 pc=0x433237
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.runtime_pollWait(0x7f497861cf08, 0x72)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/netpoll.go:306 +0x89 fp=0xc000349680 sp=0xc000349660 pc=0x464ec9
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.(*pollDesc).wait(0xc000024880?, 0x4?, 0x0)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0003496a8 sp=0xc000349680 pc=0x4d3712
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.(*pollDesc).waitRead(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/internal/poll/fd_poll_runtime.go:89
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.(*FD).Accept(0xc000024880)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc000349750 sp=0xc0003496a8 pc=0x4d901d
May 29 16:28:15  dnsbl-exporter[9388]: net.(*netFD).accept(0xc000024880)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/fd_unix.go:172 +0x35 fp=0xc000349808 sp=0xc000349750 pc=0x5b72d5
May 29 16:28:15  dnsbl-exporter[9388]: net.(*TCPListener).accept(0xc000010780)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/tcpsock_posix.go:148 +0x25 fp=0xc000349830 sp=0xc000349808 pc=0x5cc925
May 29 16:28:15  dnsbl-exporter[9388]: net.(*TCPListener).Accept(0xc000010780)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/tcpsock.go:297 +0x3d fp=0xc000349860 sp=0xc000349830 pc=0x5cba1d
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*onceCloseListener).Accept(0xc0006ceab0?)
May 29 16:28:15  dnsbl-exporter[9388]:         <autogenerated>:1 +0x2a fp=0xc000349878 sp=0xc000349860 pc=0x6c468a
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*Server).Serve(0xc000220000, {0xa62630, 0xc000010780})
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:3059 +0x385 fp=0xc0003499a8 sp=0xc000349878 pc=0x6b3065
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*Server).ListenAndServe(0xc000220000)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2988 +0x7d fp=0xc0003499d8 sp=0xc0003499a8 pc=0x6b2c9d
May 29 16:28:15  dnsbl-exporter[9388]: net/http.ListenAndServe(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:3242
May 29 16:28:15  dnsbl-exporter[9388]: github.com/Luzilla/dnsbl_exporter/app.(*DNSBLApp).Bootstrap.func1(0xc0001fa160?)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/work/dnsbl_exporter/dnsbl_exporter/app/app.go:193 +0xc05 fp=0xc000349c18 sp=0xc0003499d8 pc=0x8a9345
May 29 16:28:15  dnsbl-exporter[9388]: github.com/urfave/cli/v2.(*Command).Run(0xc0001fa160, 0xc0001922c0, {0xc0000b0120, 0x6, 0x6})
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.1/command.go:274 +0x9eb fp=0xc000349eb8 sp=0xc000349c18 pc=0x89736b
May 29 16:28:15  dnsbl-exporter[9388]: github.com/urfave/cli/v2.(*App).RunContext(0xc0001f6000, {0xa62c20?, 0xc0000b4000}, {0xc0000b0120, 0x6, 0x6})
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.1/app.go:332 +0x616 fp=0xc000349f28 sp=0xc000349eb8 pc=0x894176
May 29 16:28:15  dnsbl-exporter[9388]: github.com/urfave/cli/v2.(*App).Run(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/urfave/cli/v2@v2.25.1/app.go:309
May 29 16:28:15  dnsbl-exporter[9388]: github.com/Luzilla/dnsbl_exporter/app.(*DNSBLApp).Run(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/work/dnsbl_exporter/dnsbl_exporter/app/app.go:203
May 29 16:28:15  dnsbl-exporter[9388]: main.main()
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/work/dnsbl_exporter/dnsbl_exporter/dnsbl_exporter.go:18 +0x75 fp=0xc000349f80 sp=0xc000349f28 pc=0x8a96f5
May 29 16:28:15  dnsbl-exporter[9388]: runtime.main()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:250 +0x207 fp=0xc000349fe0 sp=0xc000349f80 pc=0x43a4a7
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000349fe8 sp=0xc000349fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 2 [force gc (idle), 14 minutes]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000042fb0 sp=0xc000042f90 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goparkunlock(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:387
May 29 16:28:15  dnsbl-exporter[9388]: runtime.forcegchelper()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:305 +0xb0 fp=0xc000042fe0 sp=0xc000042fb0 pc=0x43a710
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000042fe8 sp=0xc000042fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.init.6
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:293 +0x25
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 3 [GC sweep wait]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0xdd1501?, 0x0?, 0x0?, 0x0?, 0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000043780 sp=0xc000043760 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goparkunlock(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:387
May 29 16:28:15  dnsbl-exporter[9388]: runtime.bgsweep(0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgcsweep.go:319 +0xde fp=0xc0000437c8 sp=0xc000043780 pc=0x425c1e
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcenable.func1()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:178 +0x26 fp=0xc0000437e0 sp=0xc0000437c8 pc=0x41aea6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000437e8 sp=0xc0000437e0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcenable
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:178 +0x6b
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 4 [GC scavenge wait]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x114912146c635?, 0xb4e633?, 0x0?, 0x0?, 0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000043f70 sp=0xc000043f50 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goparkunlock(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:387
May 29 16:28:15  dnsbl-exporter[9388]: runtime.(*scavengerState).park(0xdd1a00)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000043fa0 sp=0xc000043f70 pc=0x423b13
May 29 16:28:15  dnsbl-exporter[9388]: runtime.bgscavenge(0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgcscavenge.go:633 +0x65 fp=0xc000043fc8 sp=0xc000043fa0 pc=0x424105
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcenable.func2()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:179 +0x26 fp=0xc000043fe0 sp=0xc000043fc8 pc=0x41ae46
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000043fe8 sp=0xc000043fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcenable
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:179 +0xaa
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 17 [finalizer wait, 14 minutes]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x43ac52?, 0x7f4978615728?, 0x0?, 0x0?, 0xc000042770?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000042628 sp=0xc000042608 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.runfinq()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mfinal.go:193 +0x107 fp=0xc0000427e0 sp=0xc000042628 pc=0x419ee7
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000427e8 sp=0xc0000427e0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.createfing
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mfinal.go:163 +0x45
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 21 [select]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0xc0001391a8?, 0x2?, 0xc?, 0x7b?, 0xc000138fc4?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000138e00 sp=0xc000138de0 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.selectgo(0xc0001391a8, 0xc000138fc0, 0x2?, 0x0, 0x0?, 0x1)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/select.go:327 +0x7be fp=0xc000138f40 sp=0xc000138e00 pc=0x44a09e
May 29 16:28:15  dnsbl-exporter[9388]: github.com/prometheus/client_golang/prometheus.(*Registry).Gather(0xc00006e280)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/registry.go:522 +0x9f1 fp=0xc000139420 sp=0xc000138f40 pc=0x7d6851
May 29 16:28:15  dnsbl-exporter[9388]: github.com/prometheus/client_golang/prometheus.Gatherers.Gather({0xc000064960, 0x2, 0x48489e?})
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/registry.go:753 +0x166 fp=0xc000139788 sp=0xc000139420 pc=0x7d7ec6
May 29 16:28:15  dnsbl-exporter[9388]: github.com/prometheus/client_golang/prometheus.(*Gatherers).Gather(0x4528a6?)
May 29 16:28:15  dnsbl-exporter[9388]:         <autogenerated>:1 +0x38 fp=0xc0001397b0 sp=0xc000139788 pc=0x7e1138
May 29 16:28:15  dnsbl-exporter[9388]: github.com/prometheus/client_golang/prometheus.(*noTransactionGatherer).Gather(0x4137ba?)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/registry.go:1073 +0x22 fp=0xc0001397c8 sp=0xc0001397b0 pc=0x7da662
May 29 16:28:15  dnsbl-exporter[9388]: github.com/prometheus/client_golang/prometheus/promhttp.HandlerForTransactional.func1({0xa62810, 0xc00021e1c0}, 0xc00007e100)
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/promhttp/http.go:135 +0xfe fp=0xc000139a58 sp=0xc0001397c8 pc=0x7e28fe
May 29 16:28:15  dnsbl-exporter[9388]: net/http.HandlerFunc.ServeHTTP(0x7f4978621bb8?, {0xa62810?, 0xc00021e1c0?}, 0x40da68?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2122 +0x2f fp=0xc000139a80 sp=0xc000139a58 pc=0x6af9af
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*ServeMux).ServeHTTP(0x0?, {0xa62810, 0xc00021e1c0}, 0xc00007e100)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2500 +0x149 fp=0xc000139ad0 sp=0xc000139a80 pc=0x6b1349
May 29 16:28:15  dnsbl-exporter[9388]: net/http.serverHandler.ServeHTTP({0xc0006af770?}, {0xa62810, 0xc00021e1c0}, 0xc00007e100)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2936 +0x316 fp=0xc000139b80 sp=0xc000139ad0 pc=0x6b2af6
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*conn).serve(0xc0006ceab0, {0xa62c90, 0xc00002f110})
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:1995 +0x612 fp=0xc000139fb8 sp=0xc000139b80 pc=0x6ae4d2
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*Server).Serve.func3()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:3089 +0x2e fp=0xc000139fe0 sp=0xc000139fb8 pc=0x6b344e
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000139fe8 sp=0xc000139fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by net/http.(*Server).Serve
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:3089 +0x5ed
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 258 [GC worker (idle)]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x11491214c1516?, 0x2?, 0x75?, 0xe6?, 0xc000027250?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000248750 sp=0xc000248730 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcBgMarkWorker()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1275 +0xf1 fp=0xc0002487e0 sp=0xc000248750 pc=0x41cc11
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0002487e8 sp=0xc0002487e0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcBgMarkStartWorkers
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1199 +0x25
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 257 [GC worker (idle)]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x11491214d3f6d?, 0x2?, 0x9?, 0x19?, 0xc000027250?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000247750 sp=0xc000247730 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcBgMarkWorker()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1275 +0xf1 fp=0xc0002477e0 sp=0xc000247750 pc=0x41cc11
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0002477e8 sp=0xc0002477e0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcBgMarkStartWorkers
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1199 +0x25
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 163 [GC worker (idle)]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x11491214dad90?, 0x2?, 0x88?, 0x0?, 0xc00002ad38?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc0002b6750 sp=0xc0002b6730 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcBgMarkWorker()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1275 +0xf1 fp=0xc0002b67e0 sp=0xc0002b6750 pc=0x41cc11
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0002b67e8 sp=0xc0002b67e0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcBgMarkStartWorkers
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1199 +0x25
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 20 [GC worker (idle), 8 minutes]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x1141d4e32d1aa?, 0xa?, 0xc0?, 0x45?, 0xc000027250?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000248f50 sp=0xc000248f30 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcBgMarkWorker()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1275 +0xf1 fp=0xc000248fe0 sp=0xc000248f50 pc=0x41cc11
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000248fe8 sp=0xc000248fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcBgMarkStartWorkers
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1199 +0x25
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 259 [GC worker (idle), 2 minutes]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x11471b37fbadf?, 0x2?, 0x61?, 0xa8?, 0xc000027240?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000374f50 sp=0xc000374f30 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcBgMarkWorker()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1275 +0xf1 fp=0xc000374fe0 sp=0xc000374f50 pc=0x41cc11
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000374fe8 sp=0xc000374fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcBgMarkStartWorkers
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1199 +0x25
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 26323 [IO wait]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x402020203020303?, 0xb?, 0x0?, 0x0?, 0x9?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000097da0 sp=0xc000097d80 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.netpollblock(0x4b7fe5?, 0x40482f?, 0x0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/netpoll.go:527 +0xf7 fp=0xc000097dd8 sp=0xc000097da0 pc=0x433237
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.runtime_pollWait(0x7f4978598dd8, 0x72)
May 29 16:28:15  systemd[1]: rbl_exporter.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/netpoll.go:306 +0x89 fp=0xc000097df8 sp=0xc000097dd8 pc=0x464ec9
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.(*pollDesc).wait(0xc00016fd80?, 0xc0006af781?, 0x0)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000097e20 sp=0xc000097df8 pc=0x4d3712
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.(*pollDesc).waitRead(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/internal/poll/fd_poll_runtime.go:89
May 29 16:28:15  dnsbl-exporter[9388]: internal/poll.(*FD).Read(0xc00016fd80, {0xc0006af781, 0x1, 0x1})
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc000097eb8 sp=0xc000097e20 pc=0x4d4af9
May 29 16:28:15  dnsbl-exporter[9388]: net.(*netFD).Read(0xc00016fd80, {0xc0006af781?, 0xc00057cbc0?, 0x8500a0?})
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/fd_posix.go:55 +0x29 fp=0xc000097f00 sp=0xc000097eb8 pc=0x5b5149
May 29 16:28:15  dnsbl-exporter[9388]: net.(*conn).Read(0xc00032c318, {0xc0006af781?, 0xc?, 0x8f45c0?})
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/net.go:183 +0x45 fp=0xc000097f48 sp=0xc000097f00 pc=0x5c3ae5
May 29 16:28:15  dnsbl-exporter[9388]: net.(*TCPConn).Read(0x3?, {0xc0006af781?, 0x8f45c0?, 0xc00002ac78?})
May 29 16:28:15  dnsbl-exporter[9388]:         <autogenerated>:1 +0x29 fp=0xc000097f78 sp=0xc000097f48 pc=0x5d5c49
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*connReader).backgroundRead(0xc0006af770)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:674 +0x3f fp=0xc000097fc8 sp=0xc000097f78 pc=0x6a821f
May 29 16:28:15  dnsbl-exporter[9388]: net/http.(*connReader).startBackgroundRead.func2()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:670 +0x26 fp=0xc000097fe0 sp=0xc000097fc8 pc=0x6a8146
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by net/http.(*connReader).startBackgroundRead
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:670 +0xca
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 164 [GC worker (idle)]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x11491214da650?, 0x3?, 0xb5?, 0x4f?, 0x13?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc0002b6f50 sp=0xc0002b6f30 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gcBgMarkWorker()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1275 +0xf1 fp=0xc0002b6fe0 sp=0xc0002b6f50 pc=0x41cc11
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()

May 29 16:28:15  systemd[1]: rbl_exporter.service: Failed with result 'exit-code'.
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0002b6fe8 sp=0xc0002b6fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by runtime.gcBgMarkStartWorkers
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/mgc.go:1199 +0x25
May 29 16:28:15  dnsbl-exporter[9388]: goroutine 26325 [semacquire]:
May 29 16:28:15  dnsbl-exporter[9388]: runtime.gopark(0x2?, 0x2?, 0x20?, 0x7?, 0xc000513aa0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:381 +0xd6 fp=0xc000119ef0 sp=0xc000119ed0 pc=0x43a8d6
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goparkunlock(...)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/proc.go:387
May 29 16:28:15  dnsbl-exporter[9388]: runtime.semacquire1(0xc0000b5378, 0xc0?, 0x1, 0x0, 0x3?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/sema.go:160 +0x20f fp=0xc000119f58 sp=0xc000119ef0 pc=0x44b18f
May 29 16:28:15  dnsbl-exporter[9388]: sync.runtime_Semacquire(0xc00002acc0?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/sema.go:62 +0x27 fp=0xc000119f90 sp=0xc000119f58 pc=0x4666e7
May 29 16:28:15  dnsbl-exporter[9388]: sync.(*WaitGroup).Wait(0x10?)
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/sync/waitgroup.go:116 +0x4b fp=0xc000119fb8 sp=0xc000119f90 pc=0x485b4b
May 29 16:28:15  dnsbl-exporter[9388]: github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func2()
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/registry.go:472 +0x2f fp=0xc000119fe0 sp=0xc000119fb8 pc=0x7d6d0f
May 29 16:28:15  dnsbl-exporter[9388]: runtime.goexit()
May 29 16:28:15  dnsbl-exporter[9388]:         /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000119fe8 sp=0xc000119fe0 pc=0x46a341
May 29 16:28:15  dnsbl-exporter[9388]: created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
May 29 16:28:15  dnsbl-exporter[9388]:         /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.15.1/prometheus/registry.go:471 +0x61b
till commented 1 year ago

@kozicpetar Ok, this is helpful.

The error messages in the log:

ERROR msg="" area=metrics target=mail.mymailserver

What are they about? Or is msg really empty? Otherwise, can you share the msg without your target, etc.?

kozicpetar commented 1 year ago

msg is really empty for all targets.

kozicpetar commented 1 year ago

That looks like:

May 29 16:09:04 dnsbl-exporter[8889]: time=2023-05-29T16:09:04.824+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:04 dnsbl-exporter[8889]: time=2023-05-29T16:09:04.824+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:05 dnsbl-exporter[8889]: time=2023-05-29T16:09:05.538+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:05 dnsbl-exporter[8889]: time=2023-05-29T16:09:05.538+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:08 dnsbl-exporter[8889]: time=2023-05-29T16:09:08.335+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:08 dnsbl-exporter[8889]: time=2023-05-29T16:09:08.335+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:09 dnsbl-exporter[8889]: time=2023-05-29T16:09:09.022+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:09 dnsbl-exporter[8889]: time=2023-05-29T16:09:09.022+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:10 dnsbl-exporter[8889]: time=2023-05-29T16:09:10.159+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:11 dnsbl-exporter[8889]: time=2023-05-29T16:09:11.029+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:11 dnsbl-exporter[8889]: time=2023-05-29T16:09:11.029+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:12 dnsbl-exporter[8889]: time=2023-05-29T16:09:12.166+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:12 dnsbl-exporter[8889]: time=2023-05-29T16:09:12.166+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:13 dnsbl-exporter[8889]: time=2023-05-29T16:09:13.505+02:00 level=ERROR msg="" area=metrics target=my_targets
May 29 16:09:13 dnsbl-exporter[8889]: time=2023-05-29T16:09:13.505+02:00 level=ERROR msg="" area=metrics target=my_targets
till commented 1 year ago

@kozicpetar Thank you! I'll take a look later. That should give us a hint as to why. Probably missed something when I replaced logrus.

kozicpetar commented 1 year ago

Thank you very much @till Keep waiting on update to implement this exporter. Great job !

till commented 1 year ago

@kozicpetar I made #160 to display the actual error message. Check out the branch and use make build to create an executable (on your system). Assuming you build on linux, you will also get a binary for linux.

I am still not sure why that would lead to a crash though, but maybe it helps us get further.

till commented 1 year ago

And maybe add --log.debug so we see a little more where it dies, if it dies.

kozicpetar commented 1 year ago

@till I can't build from that branch. I got error:

 make build
goreleaser build --snapshot --single-target --clean
make: goreleaser: Command not found
Makefile:3: recipe for target 'build' failed
make: *** [build] Error 127

I have a lot of errors if I trying to install goreleaser.

till commented 1 year ago

I don't know what lots of errors are. Just install a binary from their repository?

You can also look at the config in this repo to see the flags.

kozicpetar commented 1 year ago

I solved some errors but I get this now:

 make build
goreleaser build --snapshot --single-target --clean
  • starting build...
  • loading config file                              file=.goreleaser.yml
  • building only for linux/amd64                    reason=single target is enabled
  • loading environment variables
  • getting and validating git state
    • building...                                    commit=d9439ffa83925b8bff872daa22af9e132312664f latest tag=v0.6.0
    • pipe skipped                                   reason=disabled during snapshot mode
  • parsing tag
  • setting defaults
      • DEPRECATED: `archives.rlcp` will be the default soon, check https://goreleaser.com/deprecations#archivesrlcp for more info
  • running before hooks
    • running                                        hook=go mod tidy
  ⨯ build failed after 0s                    error=hook failed: go mod tidy: exit status 2; output: go: unknown subcommand "mod"
Run 'go help' for usage.

Makefile:3: recipe for target 'build' failed
make: *** [build] Error 1

EDIT: If I run command go mod tidy I get: 

go mod tidy: go.mod file indicates go 1.20, but maximum supported version is 1.16

Do I need to update go locally to 1.20?

till commented 1 year ago

@kozicpetar Yeah, check the go.mod. If you can't update, maybe build in a container?

kozicpetar commented 1 year ago

@till thank you very much on help. I updated go to 1.20 and I just build binary. I run exporter with debug flag. We will see. Let you know about status.

kozicpetar commented 1 year ago

@till

Exporter stop wokring again, this is log:

Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.730+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=mail.mymailserver target=mail.mymailserver v=xxx.xxx.xxx.xxx
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.730+02:00 level=DEBUG msg="About to query RBL" area=metrics target=mail.mymailserver rbl=l2.spews.dnsbl.sorbs.net ip=xxx.xxx.xxx.xxx
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.730+02:00 level=DEBUG msg="Next up" area=metrics target=mail.mymailserver rbl=misc.dnsbl.sorbs.net ip=mail.mymailserver
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.731+02:00 level=DEBUG msg=Roundtrip area=metrics target=mail.mymailserver v=0.000253013
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.731+02:00 level=DEBUG msg=Roundtrip area=metrics target=mail.mymailserver v=0.00026872
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.731+02:00 level=DEBUG msg=Roundtrip area=metrics target=mail.mymailserver v=0.000221963
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.731+02:00 level=DEBUG msg=Roundtrip area=metrics target=mail.mymailserver v=0.00015271
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.731+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=mail.mymailserver target=mail.mymailserver v=xxx.xxx.xxx.xxx
Jun 02 15:15:13  dnsbl-exporter[17846]: time=2023-06-02T15:15:13.731+02:00 level=DEBUG msg="About to query RBL" area=metrics target=mail.mymailserver rbl=misc.dnsbl.sorbs.net ip=xxx.xxx.xxx.xxx
Jun 02 15:15:13  dnsbl-exporter[17846]: panic: runtime error: invalid memory address or nil pointer dereference
Jun 02 15:15:13  dnsbl-exporter[17846]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x416a52]
Jun 02 15:15:13  dnsbl-exporter[17846]: goroutine 102542 [running]:
Jun 02 15:15:13  dnsbl-exporter[17846]: github.com/Luzilla/dnsbl_exporter/pkg/rbl.(*Rbl).lookup(0xc000227560, {0xc0000c29f0, 0x13}, {0xc0000c2d20, 0x16})
Jun 02 15:15:13  dnsbl-exporter[17846]:         /root/go/dnsbl_exporter/pkg/rbl/rbl.go:127 +0x376
Jun 02 15:15:13  dnsbl-exporter[17846]: github.com/Luzilla/dnsbl_exporter/pkg/rbl.(*Rbl).Update.func1({0xc0000c29f0, 0x13}, {0xc0000c2d20, 0x16})
Jun 02 15:15:13  dnsbl-exporter[17846]:         /root/go/dnsbl_exporter/pkg/rbl/rbl.go:56 +0x1f6
Jun 02 15:15:13  dnsbl-exporter[17846]: created by github.com/Luzilla/dnsbl_exporter/pkg/rbl.(*Rbl).Update
Jun 02 15:15:13  dnsbl-exporter[17846]:         /root/go/dnsbl_exporter/pkg/rbl/rbl.go:51 +0x50
Jun 02 15:15:13  systemd[1]: rbl_exporter.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun 02 15:15:13  systemd[1]: rbl_exporter.service: Failed with result 'exit-code'.
Jun 02 15:15:13  systemd[1]: rbl_exporter.service: Consumed 16.528s CPU time.
till commented 1 year ago

@kozicpetar thanks, I'll take a look later. I think this is something in the response of the DNS server that I am not handling correctly.

kozicpetar commented 1 year ago

@till any chance to solve this soon? Thank you.

till commented 1 year ago

@kozicpetar Tomorrow/Saturday.

till commented 1 year ago

@kozicpetar hey, I wasn't able to replicate the error but I used the opportunity to make additional changes and to add test cases.

I think the DNS lookup fails somewhere, but I think I may have covered it now. If you want to try again.

Otherwise, if it still fails. Maybe let's get in touch on Twitter or Mastodon so you can give me more config and I'll have a look again? My links are on my profile.

kozicpetar commented 1 year ago

@till thank you very much. I will test again today.

kozicpetar commented 1 year ago

Unfortunately, same again.

This is error:

Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Built lookup" area=metrics target=e4.my_mail_server lookup=xxx.xxx.xxx.xxx7.dyna.spamrats.com
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Next up" area=metrics target=e4.my_mail_server rbl=pbl.spamhaus.org ip=e4.my_mail_server
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Next up" area=metrics target=e4.my_mail_server rbl=ips.backscatterer.org ip=e4.my_mail_server
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Next up" area=metrics target=e4.my_mail_server rbl=dul.dnsbl.sorbs.net ip=e4.my_mail_server
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000728477
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=e4.my_mail_server target=e4.my_mail_server v=xxx.xxx.xxx.xxx
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Next up" area=metrics target=e4.my_mail_server rbl=sbl.spamhaus.org ip=e4.my_mail_server
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000254739
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=e4.my_mail_server target=e4.my_mail_server v=xxx.xxx.xxx.xxx
Jun 09 10:09:29 systemd[1]: rbl_exporter.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Next up" area=metrics target=e4.my_mail_server rbl=drone.abuse.ch ip=e4.my_mail_server
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="About to query RBL" area=metrics target=e4.my_mail_server rbl=rbl.schulte.org ip=xxx.xxx.xxx.xxx7
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Built lookup" area=metrics target=e4.my_mail_server lookup=xxx.xxx.xxx.xxx7.rbl.schulte.org
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000220778
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000241866
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=e4.my_mail_server target=e4.my_mail_server v=xxx.xxx.xxx.xxx
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="About to query RBL" area=metrics target=e4.my_mail_server rbl=dnsbl.sorbs.net ip=xxx.xxx.xxx.xxx7
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Built lookup" area=metrics target=e4.my_mail_server lookup=xxx.xxx.xxx.xxx7.dnsbl.sorbs.net
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000264348
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.0002883
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.908+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=e4.my_mail_server target=e4.my_mail_server v=xxx.xxx.xxx.xxx
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Next up" area=metrics target=e4.my_mail_server rbl=orvedb.aupads.org ip=e4.my_mail_server
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.908+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000442671
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="Next up" area=metrics target=e4.my_mail_server rbl=spam.spamrats.com ip=e4.my_mail_server
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.908+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000482851
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=e4.my_mail_server target=e4.my_mail_server v=xxx.xxx.xxx.xxx
Jun 09 10:09:29 systemd[1]: rbl_exporter.service: Failed with result 'exit-code'.
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.908+02:00 level=DEBUG msg=Roundtrip area=metrics target=e4.my_mail_server v=0.000517245
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="About to query RBL" area=metrics target=e4.my_mail_server rbl=smtp.dnsbl.sorbs.net ip=xxx.xxx.xxx.xxx7
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.908+02:00 level=DEBUG msg="Built lookup" area=metrics target=e4.my_mail_server lookup=xxx.xxx.xxx.xxx7.smtp.dnsbl.sorbs.net
Jun 09 10:09:29 dnsbl-exporter[2731]: time=2023-06-09T10:09:29.907+02:00 level=DEBUG msg="We have an A-Record" area=metrics target=e4.my_mail_server target=e4.my_mail_server v=xxx.xxx.xxx.xxx
Jun 09 10:09:29 dnsbl-exporter[2731]: panic: runtime error: invalid memory address or nil pointer dereference
Jun 09 10:09:29 dnsbl-exporter[2731]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x120 pc=0x46cdcc]
Jun 09 10:09:29 dnsbl-exporter[2731]: goroutine 99075 [running]:
Jun 09 10:09:29 dnsbl-exporter[2731]: github.com/Luzilla/dnsbl_exporter/pkg/rbl.(*Rbl).lookup(0xc000410330, {0xc0000b65d0, 0x10}, {0xc0000b2ec0, 0xa})
Jun 09 10:09:29 dnsbl-exporter[2731]:         /root/go/dnsbl_exporter/pkg/rbl/rbl.go:138 +0x3ef
Jun 09 10:09:29 dnsbl-exporter[2731]: github.com/Luzilla/dnsbl_exporter/pkg/rbl.(*Rbl).Update.func1({0xc0000b65d0, 0x10}, {0xc0000b2ec0, 0xa})
Jun 09 10:09:29 dnsbl-exporter[2731]:         /root/go/dnsbl_exporter/pkg/rbl/rbl.go:55 +0x1f6
Jun 09 10:09:29 dnsbl-exporter[2731]: created by github.com/Luzilla/dnsbl_exporter/pkg/rbl.(*Rbl).Update
Jun 09 10:09:29 dnsbl-exporter[2731]:         /root/go/dnsbl_exporter/pkg/rbl/rbl.go:50 +0x50
Jun 09 10:09:29 systemd[1]: rbl_exporter.service: Consumed 16.101s CPU time.

Unbound service is ok.

Dig works as expected:

dig +short @127.0.0.1 spamhaus.org
3.126.246.180
till commented 1 year ago

@kozicpetar I responded on Friday or so, lmk

till commented 1 year ago

I am trying to clean-up the code a little and I am letting it run locally with watch -n 10 curl http://localhost:9211/metrics. It's been running for a couple hours. No crashes. I'll push it too later tonight.

kozicpetar commented 1 year ago

@till Thank you on update. I compile yesterday and exporter working without problem from yesterday. We will see. I will keep you updated.

till commented 1 year ago

I'll keep it open for a bit, can you let me know next week if it continues to work?

kozicpetar commented 1 year ago

HI @till sure I let you know. For now, working as expected.

kozicpetar commented 1 year ago

@till Exporter working without problems last 5 days. Thank you !

kozicpetar commented 11 months ago

Hi @till did you merge changes to main and new release? Thank you.

till commented 10 months ago

I can try to do that some time this week or next.

till commented 7 months ago

@kozicpetar I merged to main finally. Do you want to rebuild and test again before I cut a release? I am trying to get some docs in before anyway. I think ideally, within November.

kozicpetar commented 7 months ago

It’s Petar’s wife,he died on october the 2.Sent from my iPhoneOn 14. 11. 2023., at 15:21, Till! @.***> wrote: @kozicpetar I merged to main finally. Do you want to rebuild and test again before I cut a release? I am trying to get some docs in before anyway. I think ideally, within November.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

till commented 7 months ago

@kozicpetar I am sorry to hear this, and my condolences to you.

Just in case you are not aware, your response ended up on a public forum/platform. Just in case there's similar email coming through.

Till