search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
717 stars 1.38k forks source link

snmpd core dump #6940

Open qiluo-msft opened 3 years ago

qiluo-msft commented 3 years ago

Description

Steps to reproduce the issue:

snmpd crashed and auto recovered.

Use gdb to check the stack

Reading symbols from /usr/sbin/snmpd...(no debugging symbols found)...done.
[New LWP 27]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/snmpd -f -LS4d -u Debian-snmp -g Debian-snmp -I -smux mteTrigger mteT'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f0e7673e067 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0  0x00007f0e7673e067 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f0e7673f448 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f0e7677c1b4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007f0e7678198e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x00007f0e76781c14 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x00007f0e76782e98 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#6  0x00007f0e76785020 in malloc () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x00007f0e77019a62 in _sess_read () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30
#8  0x00007f0e7701a579 in snmp_sess_read2 () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30
#9  0x00007f0e7701a5cb in snmp_read2 () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30
#10 0x000000000040483f in main ()

Describe the results you received:

Describe the results you expected:

Output of show version:

SONiC.20181130.70

Additional information you deem important (e.g. issue happens only occasionally):

ghost commented 3 years ago

@qiluo-msft, what is the current status of the issue? I have tried to catch a crash of snmpd on https://sonic-jenkins.westus2.cloudapp.azure.com/job/mellanox/job/buildimage-mlnx-201811/226/ and it seems there are no crashes since start of the system. Could you provide some additional information about how to reproduce the crashes?

qiluo-msft commented 3 years ago

To me it is random and rare. We did not make progress on this topic yet.

ghost commented 3 years ago

@qiluo-msft, I have manually installed gdb into the docker container and attached to snmpd process. Tested in this case with different snmp clients. snmp works fine, no crashes occurred in 2 last days. It would be good to check diff between current state of the branch and state of SONiC.20181130.70. Unfortunately, already there is no such build on Jenkins. Have you information regarding build date for the build or something else? Also, it will be good to know how rare the issue has been occurred, approximately?