Open lerry-lee opened 1 year ago
PR #13669 is a hotfix to unblock some PR tests.
Hi @lerry-lee Did you see the traceback from the syslog when the issue occurred? I observed a similar issue #13791 which I also suspect sonic-cfggen aborted somehow but I didn't see the traceback from the log.
Hi @lerry-lee Did you see the traceback from the syslog when the issue occurred? I observed a similar issue #13791 which I also suspect sonic-cfggen aborted somehow but I didn't see the traceback from the log. @stephenxs After issue occurred, I checked the syslog and saw the error msg like the screenshot in the top description. I saw your observed issue also related to
sonic-cfggen -d
, I think it might be some related cause, because-d
requires Redis, so I think some problems occurred when getting connection to Redis.
Hi @lerry-lee Did you see the traceback from the syslog when the issue occurred? I observed a similar issue #13791 which I also suspect sonic-cfggen aborted somehow but I didn't see the traceback from the log. @stephenxs After issue occurred, I checked the syslog and saw the error msg like the screenshot in the top description. I saw your observed issue also related to
sonic-cfggen -d
, I think it might be some related cause, because-d
requires Redis, so I think some problems occurred when getting connection to Redis.
Thanks.
I also suspect they share the same cause. The only difference is that I didn't see the traceback from the dumps.
I'm going to try using -a
to avoid redis db access
can we run the tcpdump on lo interface when the team starts to check the packet dump on why the connection to redis is failing.
please update the description with the testcase where this failure is seen.
please update the description with the testcase where this failure is seen.
Refined.
Description
In
t0-sonic
andmulti-asic
KVM-test, sometimestest_pretest.py::test_disable_rsyslog_rate_limit
will fail. Error message like below:The reason is that rsyslogd process can not enter running state because its' rsyslog.conf is empty. Further, we found that, in teamd.sh, when start
teamd
, it will update rsyslog.conf usesonic-cfggen
, related code is below:And if this command fails, it will generate an empty file and copy it to
teamd
container. Currently, it uses-d
parameter which means read from configDB. We see thesonic-cfggen
failed reason is:But in the aspect of the service dependency control, we found that
teamd
will start afterdatabase
. So, the connection to Redis meets some problems.Steps to reproduce the issue:
1. 2. 3.
Describe the results you received:
Describe the results you expected:
Output of
show version
:Output of
show techsupport
:Additional information you deem important (e.g. issue happens only occasionally):