panic: runtime error: invalid memory address or nil pointer dereference

[theobarrague]

Describe the bug Something bad happened during a memory access

To Reproduce ¯\_(ツ)_/¯

Expected behavior Run forever without memory exception

OS (please complete the following information):

• OS: Debian
• Version 10

Additional context

prometheus-squid-exporter[75168]: panic: runtime error: invalid memory address or nil pointer dereference
prometheus-squid-exporter[75168]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x75cc20]
prometheus-squid-exporter[75168]: goroutine 2221 [running]:
prometheus-squid-exporter[75168]: github.com/boynux/squid-exporter/collector.(*CacheObjectClient).GetCounters(0xc000085800, 0x0, 0x85fc2a, 0x5, 0xc00
prometheus-squid-exporter[75168]:         github.com/boynux/squid-exporter/collector/client.go:65 +0xf0
prometheus-squid-exporter[75168]: github.com/boynux/squid-exporter/collector.(*Exporter).Collect(0xc0000858f0, 0xc0000902a0)
prometheus-squid-exporter[75168]:         github.com/boynux/squid-exporter/collector/metrics.go:62 +0x69
prometheus-squid-exporter[75168]: github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
prometheus-squid-exporter[75168]:         github.com/prometheus/client_golang/prometheus/registry.go:430 +0x193
prometheus-squid-exporter[75168]: created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
prometheus-squid-exporter[75168]:         github.com/prometheus/client_golang/prometheus/registry.go:522 +0xe23
systemd[1]: prometheus-squid-exporter.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
systemd[1]: prometheus-squid-exporter.service: Failed with result 'exit-code'.

[boynux]

Hi, Thank you for reporting this. I'll take a look. To make testing easier can you provide some additional information?

Which version of exporter do you use? Which version of squid? What are the arguments you use for the exporter? Do you use authentication features of the exporter? How often your Prometheus instance fetches the metrics from the exporter and typically how long it takes before the crash?

[theobarrague]

Which version of exporter do you use? 1.4 Which version of squid? 4.6 What are the arguments you use for the exporter? None Do you use authentication features of the exporter? No How often your Prometheus instance fetches the metrics from the exporter and typically how long it takes before the crash? Every 30 seconds and take ~ 5 ms

$ prometheus-squid-exporter -version
2022/02/07 11:06:32 squid_exporter, version 1.4+ds (branch: debian/sid, revision: 1.4+ds-1+b10)
  build user:       pkg-go-maintainers@lists.alioth.debian.org
  build date:       20190311-02:53:08
  go version:       go1.11.5

$ apt-cache policy prometheus-squid-exporter
prometheus-squid-exporter:
  Installed: 1.4+ds-1+b10
  Candidate: 1.4+ds-1+b10
  Version table:
 *** 1.4+ds-1+b10 500
        500 http://ftp.be.debian.org/debian buster/main amd64 Packages
        100 /var/lib/dpkg/status

$ cat /proc/54630/cmdline | sed -e "s/\x00/ /g"; echo
/usr/bin/prometheus-squid-exporter

$ squid --version
Squid Cache: Version 4.6
Service Name: squid
Debian linux
configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 -fdebug-prefix-map=/build/squid-SXQquB/squid-4.6=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -latomic' 'BUILDCXX=x86_64-linux-gnu-g++' '--with-build-environment=default' '--enable-build-info=Debian linux' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native' '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-icap-client' '--enable-follow-x-forwarded-for' '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' '--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper' '--enable-auth-ntlm=fake,SMB_LM' '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,time_quota,unix_group,wbinfo_group' '--enable-security-cert-validators=fake' '--enable-storeid-rewrite-helpers=file' '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--disable-translation' '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' '--with-gnutls' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CC=x86_64-linux-gnu-gcc' 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/squid-SXQquB/squid-4.6=. -fstack-protector-strong -Wformat -Werror=format-security -Wall' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -latomic' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXX=x86_64-linux-gnu-g++' 'CXXFLAGS=-g -O2 -fdebug-prefix-map=/build/squid-SXQquB/squid-4.6=. -fstack-protector-strong -Wformat -Werror=format-security'

Feb  2 08:31:03 *** prometheus-squid-exporter[75168]: 2022/02/02 08:31:03 Listening on :9301
Feb  2 09:53:28 *** prometheus-squid-exporter[75168]: panic: runtime error: invalid memory address or nil pointer dereference
...
Feb  2 10:12:13 *** prometheus-squid-exporter[115840]: 2022/02/02 10:12:13 Listening on :9301
Feb  2 10:23:55 *** prometheus-squid-exporter[115840]: panic: runtime error: invalid memory address or nil pointer dereference

[boynux]

Thanks, I tried to debug the issue but unfortunately I could not reproduce the problem. I think you are using the the exporter version from the debian package maintainers. This is an old build of this project amd it's possible that the bug is already fixed.

Could you either build the binary from main branch or download the latest binary form the releases page and give it a try?

If you are able to reproduce the issue the stack trace will have updated code references which is much more helpful to debug the issue.

[boynux]

btw, I suspect this commit https://github.com/boynux/squid-exporter/commit/3db62bc68eadaec507156216cd21af307ca8971b is a fix to the issue that you are observing. It is included in 1.8.3 version and above.

[theobarrague]

I'm testing with the version v1.10.0, no doubt that it will works now. I was influenced by the output of -version which show the current date and not the date of last build ...

[boynux]

Thanks for the update, I'll go ahead and close this issue. If you encountered this problem again with the recent releases please reopen it with the new stack trace and I'll take a look.