sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
730 stars 1.4k forks source link

[dhcp_server] [rsyslog] Logs are not written to host #18472

Closed vivekrnv closed 3 months ago

vivekrnv commented 6 months ago

Description

dhcp_server rsyslog container plugin is not writing logs to host. Logs are written to /var/log/syslog under the container

Steps to reproduce the issue:

  1. Run https://github.com/sonic-net/sonic-mgmt/blob/master/tests/process_monitoring/test_critical_process_monitoring.py to easily repro the issue
  2. or start the dhcp_server and see nothing is written to syslog

Describe the results you received:

Expected Messages that are missing:
.*Process 'dhcpservd' is not running in namespace 'host'.*
.*Process 'kea-dhcp4' is not running in namespace 'host'.*

Describe the results you expected:

Logs should be written correctly

vivekrnv commented 6 months ago

@yxieca, @yaqiangz PFA

yaqiangz commented 6 months ago

@yxieca, @yaqiangz PFA

@vivekrnv It's a known issue, dhcp_server container is running on bridge network mode. For now, syslog of it can be found in /etc/log/syslog inside container

prgeor commented 6 months ago

@yaqiangz this is not expected behavior. All the dhcp_server logs must be logged on the host /var/log because

  1. We need log rotation to work which runs in the host, in order to avoid disk getting full overtime
  2. Also this may cause disk wear out over time

@Blueve Can you please prioritize this P0 MUST fix

Blueve commented 6 months ago

@prgeor we might not have enough capacity until June.

@vivekrnv can you purpose a fix if this is urgent? we can help review.

vivekrnv commented 6 months ago

@Blueve, I currently don't have BW to work on this. Please don't close the issue as the problem still exists and also the test process_monitoring/test_critical_process_monitoring.py fails when the dhcp server is enabled

Blueve commented 5 months ago

Update ETA: 30 June

yaqiangz commented 3 months ago

https://github.com/sonic-net/sonic-buildimage/pull/19303 Fixed @vivekrnv for viz