search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
723 stars 1.38k forks source link

[lldp][rsyslog]Sometimes during the bootup, there are no syslogs from lldp container #15152

Open dgsudharsan opened 1 year ago

dgsudharsan commented 1 year ago

Description

Hit this issue once. On bootup after the container initialization, there should be syslogs from lldp container which will show the process spawned. However in one specific bootup after fresh installation, there were no syslogs. Config reload or reboot didn't help to recover.

Attached is the syslog file during reboot.

Steps to reproduce the issue:

1. 2. 3.

Describe the results you received:

Logs after lldp container start

May 16 21:19:24.872953 r-liger-02 NOTICE admin: Starting lldp service...
May 16 21:19:25.036370 r-liger-02 INFO lldp.sh[10303]: Starting existing lldp container with HWSKU ACS-MSN4600
May 16 21:19:26.372888 r-liger-02 DEBUG container: read_data: config:True feature:lldp fields:[('set_owner', 'local'), ('no_fallback_to_local', False), ('state', 'disabled')] val:['local', False, 'enabled']
May 16 21:19:26.374290 r-liger-02 DEBUG container: read_data: config:False feature:lldp fields:[('current_owner', 'none'), ('remote_state', 'none'), ('container_id', '')] val:['none', 'none', '']
May 16 21:19:26.374400 r-liger-02 DEBUG container: container_wait: lldp: set_owner:local ct_owner:none state:none id:lldp pend=0
May 16 21:19:26.374603 r-liger-02 DEBUG container: container_wait: END -- transitioning to docker wait
..
May 16 21:19:31.440319 r-liger-02 INFO hostcfgd: Updating feature 'lldp' systemd config file related to auto-restart ...
May 16 21:19:31.440537 r-liger-02 INFO hostcfgd: Feautre 'lldp' systemd config file related to auto-restart is update
May 16 21:19:31.440620 r-liger-02 INFO hostcfgd: Reloading systemd configuration files ...
May 16 21:19:31.455164 r-liger-02 INFO systemd[1]: Reloading.

Describe the results you expected:

May 19 07:20:58.819160 r-liger-02 INFO container: docker cmd: start for lldp
May 19 07:20:58.902668 r-liger-02 NOTICE root: Starting lldp service...
May 19 07:20:59.030264 r-liger-02 NOTICE admin: Started lldp service...
May 19 07:20:59.322646 r-liger-02 NOTICE root: Started lldp service
May 19 07:20:59.716696 r-liger-02 DEBUG container: read_data: config:True feature:lldp fields:[('set_owner', 'local'), ('no_fallback_to_local', False), ('state', 'disabled')] val:['local', False, 'enabled']
May 19 07:20:59.717401 r-liger-02 DEBUG container: read_data: config:False feature:lldp fields:[('current_owner', 'none'), ('remote_state', 'none'), ('container_id', '')] val:['none', 'none', '']
May 19 07:20:59.718488 r-liger-02 DEBUG container: container_wait: lldp: set_owner:local ct_owner:none state:none id:lldp pend=0
May 19 07:21:02.546022 r-liger-02 INFO lldp#rsyslogd: [origin software="rsyslogd" swVersion="8.2302.0" x-pid="13" x-info="https://www.rsyslog.com"] start
May 19 07:21:02.547742 r-liger-02 INFO lldp#supervisord 2023-05-19 04:20:59,914 INFO Included extra file "/etc/supervisor/conf.d/containercfgd.conf" during parsing
May 19 07:21:02.547742 r-liger-02 INFO lldp#supervisord 2023-05-19 04:20:59,914 INFO Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
May 19 07:21:02.547742 r-liger-02 INFO lldp#supervisord 2023-05-19 04:20:59,914 INFO Set uid to user 0 succeeded

Output of show version:

SONiC Software Version: SONiC.202211_RC14.2-7a20284d5_Internal
SONiC OS Version: 11
Distribution: Debian 11.7
Kernel: 5.10.0-18-2-amd64
Build commit: 7a20284d5
Build date: Tue May 16 11:28:07 UTC 2023
Built by: sw-r2d2-bot@r-build-sonic-ci02-244

Platform: x86_64-mlnx_msn4600-r0
HwSKU: ACS-MSN4600
ASIC: mellanox
ASIC Count: 1
Serial Number: MT2126X19873
Model Number: MSN4600-VS2FO_QP1
Hardware Revision: A1
Uptime: 10:30:47 up 3 min,  1 user,  load average: 1.82, 2.11, 1.01
Date: Wed 17 May 2023 10:30:47

Docker images:
REPOSITORY                                         TAG                                IMAGE ID       SIZE
docker-syncd-mlnx                                  202211_RC14.2-7a20284d5_Internal   2d390092b086   876MB
docker-syncd-mlnx                                  latest                             2d390092b086   876MB
docker-orchagent                                   202211_RC14.2-7a20284d5_Internal   30957fcad7ed   475MB
docker-orchagent                                   latest                             30957fcad7ed   475MB
docker-fpm-frr                                     202211_RC14.2-7a20284d5_Internal   1249a403b8f7   486MB
docker-fpm-frr                                     latest                             1249a403b8f7   486MB
docker-platform-monitor                            202211_RC14.2-7a20284d5_Internal   c88bd98e946e   877MB
docker-platform-monitor                            latest                             c88bd98e946e   877MB
docker-teamd                                       202211_RC14.2-7a20284d5_Internal   09803e67c958   456MB
docker-teamd                                       latest                             09803e67c958   456MB
docker-snmp                                        202211_RC14.2-7a20284d5_Internal   ad80ce837359   485MB
docker-snmp                                        latest                             ad80ce837359   485MB
docker-dhcp-relay                                  latest                             88ef2955e5f0   449MB
docker-macsec                                      latest                             05ed9029474f   458MB
docker-sonic-telemetry                             202211_RC14.2-7a20284d5_Internal   d66ee47715ad   738MB
docker-sonic-telemetry                             latest                             d66ee47715ad   738MB
docker-eventd                                      202211_RC14.2-7a20284d5_Internal   6c304697d4ee   440MB
docker-eventd                                      latest                             6c304697d4ee   440MB
docker-database                                    202211_RC14.2-7a20284d5_Internal   16a7ba051ff5   440MB
docker-database                                    latest                             16a7ba051ff5   440MB
docker-mux                                         202211_RC14.2-7a20284d5_Internal   01630e9c78a6   488MB
docker-mux                                         latest                             01630e9c78a6   488MB
docker-lldp                                        202211_RC14.2-7a20284d5_Internal   dd8f27fca8c4   482MB
docker-lldp                                        latest                             dd8f27fca8c4   482MB
docker-sonic-p4rt                                  202211_RC14.2-7a20284d5_Internal   471f79b4b984   522MB
docker-sonic-p4rt                                  latest                             471f79b4b984   522MB
docker-router-advertiser                           202211_RC14.2-7a20284d5_Internal   8cf37fac04b5   440MB
docker-router-advertiser                           latest                             8cf37fac04b5   440MB
urm.nvidia.com/sw-nbu-sws-sonic-docker/sonic-wjh   1.5.0-202211-internal-4            f79be67c5ca6   323MB
docker-sonic-mgmt-framework                        202211_RC14.2-7a20284d5_Internal   54ed482e06dc   551MB
docker-sonic-mgmt-framework                        latest                             54ed482e06dc   551MB
docker-nat                                         202211_RC14.2-7a20284d5_Internal   442727751172   423MB
docker-nat                                         latest                             442727751172   423MB
docker-sflow                                       202211_RC14.2-7a20284d5_Internal   912fbf77e6ef   421MB
docker-sflow                                       latest                             912fbf77e6ef   421MB

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

syslog.203.gz

arlakshm commented 1 year ago

issue is seen just once.