search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
738 stars 1.43k forks source link

[snmp] Snmpd fails to start when mgmt or Loopback interface is configured with Link local IPv6 address #16001

Open dgsudharsan opened 1 year ago

dgsudharsan commented 1 year ago

Description

Issue started appearing after the PR https://github.com/sonic-net/sonic-buildimage/pull/15487. This PR introduces generating agentAddress for loopback and mgmt interface. However if the mgmt or loopback interface has link local ip address, snmpd fails to start and ends up in fatal state

docker exec snmp supervisorctl status
containercfgd                    RUNNING   pid 23, uptime 0:00:46
dependent-startup                EXITED    Jul 31 05:52 AM
rsyslogd                         RUNNING   pid 18, uptime 0:00:48
snmp-subagent                    RUNNING   pid 24, uptime 0:00:46
snmpd                            RUNNING   pid 22, uptime 0:00:48
start                            EXITED    Jul 31 05:52 AM
supervisor-proc-exit-listener    RUNNING   pid 8, uptime 0:00:50

Steps to reproduce the issue:

  1. Config link local ipv6 address on mgmt or loopback interface
  2. Restart snmp docker

Describe the results you received:

snmp daemon fails to start

Describe the results you expected:

snmp daemon shouldn't fail.

Output of show version:

SONiC Software Version: SONiC.202211_1_RC2.33-8932a44a4_Internal
SONiC OS Version: 11
Distribution: Debian 11.7
Kernel: 5.10.0-23-2-amd64
Build commit: b6eda87f7
Build date: Fri Jul 28 06:15:58 UTC 2023
Built by: sw-r2d2-bot@r-build-sonic-ci03-243

Platform: x86_64-mlnx_msn3800-r0
HwSKU: Mellanox-SN3800-D112C8
ASIC: mellanox
ASIC Count: 1
Serial Number: MT2208X03836
Model Number: MSN3800-CS2FOS
Hardware Revision: A3
Uptime: 08:47:35 up 5 min,  1 user,  load average: 0.91, 0.80, 0.40
Date: Mon 31 Jul 2023 08:47:35

Docker images:
REPOSITORY                                         TAG                                  IMAGE ID       SIZE
docker-syncd-mlnx                                  202211_1_RC2.33-8932a44a4_Internal   5719b528b08b   969MB
docker-syncd-mlnx                                  latest                               5719b528b08b   969MB
docker-platform-monitor                            202211_1_RC2.33-8932a44a4_Internal   f80e4fcd45e3   964MB
docker-platform-monitor                            latest                               f80e4fcd45e3   964MB
docker-fpm-frr                                     202211_1_RC2.33-8932a44a4_Internal   c0e128b9ec43   486MB
docker-fpm-frr                                     latest                               c0e128b9ec43   486MB
docker-orchagent                                   202211_1_RC2.33-8932a44a4_Internal   ff4cf8360460   475MB
docker-orchagent                                   latest                               ff4cf8360460   475MB
docker-teamd                                       202211_1_RC2.33-8932a44a4_Internal   c8d5f20959a9   456MB
docker-teamd                                       latest                               c8d5f20959a9   456MB
docker-macsec                                      latest                               ae708db42500   458MB
docker-snmp                                        202211_1_RC2.33-8932a44a4_Internal   f59854f7e54d   484MB
docker-snmp                                        latest                               f59854f7e54d   484MB
docker-dhcp-relay                                  latest                               03f1cee626ab   449MB
docker-sonic-telemetry                             202211_1_RC2.33-8932a44a4_Internal   c45873a00691   738MB
docker-sonic-telemetry                             latest                               c45873a00691   738MB
docker-eventd                                      202211_1_RC2.33-8932a44a4_Internal   b676664aa893   439MB
docker-eventd                                      latest                               b676664aa893   439MB
docker-router-advertiser                           202211_1_RC2.33-8932a44a4_Internal   08dbf046d19f   439MB
docker-router-advertiser                           latest                               08dbf046d19f   439MB
docker-lldp                                        202211_1_RC2.33-8932a44a4_Internal   8b859fde0a0f   482MB
docker-lldp                                        latest                               8b859fde0a0f   482MB
docker-database                                    202211_1_RC2.33-8932a44a4_Internal   0ab7ff51abbd   439MB
docker-database                                    latest                               0ab7ff51abbd   439MB
docker-mux                                         202211_1_RC2.33-8932a44a4_Internal   41432898407d   488MB
docker-mux                                         latest                               41432898407d   488MB
docker-sonic-p4rt                                  202211_1_RC2.33-8932a44a4_Internal   54b0f9aac5c1   522MB
docker-sonic-p4rt                                  latest                               54b0f9aac5c1   522MB
docker-nat                                         202211_1_RC2.33-8932a44a4_Internal   74e5f0c06314   425MB
docker-nat                                         latest                               74e5f0c06314   425MB
docker-sflow                                       202211_1_RC2.33-8932a44a4_Internal   bf58a5281929   423MB
docker-sflow                                       latest                               bf58a5281929   423MB
docker-sonic-mgmt-framework                        202211_1_RC2.33-8932a44a4_Internal   849aa6ac959a   553MB
docker-sonic-mgmt-framework                        latest                               849aa6ac959a   553MB
urm.nvidia.com/sw-nbu-sws-sonic-docker/sonic-wjh   1.5.4-202211-6                       b1b4b494307e   432MB
urm.nvidia.com/sw-nbu-sws-sonic-docker/doai        1.0.0-202211-3                       7369dae9ce32   200MB

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

sonic_dump_r-tigris-26_20230731_084718.tar.gz

vivekrnv commented 9 months ago

This issue started to reappear after https://github.com/sonic-net/sonic-buildimage/pull/17045 

root@host:/# /usr/sbin/snmpd -f -LO0-7d -u Debian-snmp -g Debian-snmp -I -smux,mteTrigger,mteTriggerConf,ifTable,ifXTable,inetCidrRouteTable,ipCidrRouteTable,ip,disk_hw -p /run/snmpd.pid
Turning on AgentX master support.
Error opening specified endpoint "udp6:[fe80::ac0:ebff:fee1:4ba2]:161"
Server Exiting with code 1

Error is in src/sonic-config-engine/minigraph.py. It should check whether the IP is link local and add zone_id if so.

++ @SuvarnaMeenakshi

liat-grozovik commented 8 months ago

@SuvarnaMeenakshi any ETA for this issue?

SuvarnaMeenakshi commented 8 months ago

@SuvarnaMeenakshi any ETA for this issue?

@liat-grozovik @keboliu Plan to raise a PR this week 3/8

keboliu commented 8 months ago

Hi @SuvarnaMeenakshi have you already had a fix for this issue?

SuvarnaMeenakshi commented 8 months ago

Hi @SuvarnaMeenakshi have you already had a fix for this issue? Raised below PRs to fix: https://github.com/sonic-net/sonic-buildimage/pull/18350/ https://github.com/sonic-net/sonic-utilities/pull/3215

keboliu commented 6 months ago

@lguohan there are two PRs to fix this issue, https://github.com/sonic-net/sonic-utilities/pull/3215 is still open.