search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
730 stars 1.4k forks source link

Z9264/201911-170/NAT is not running in the latest November image #5259

Open mini-nair-dell opened 4 years ago

mini-nair-dell commented 4 years ago

Description: ============ Nat container is not running in the latest image

root@sonic:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 28bc2fb500f3 docker-sonic-telemetry:latest "/usr/bin/supervisord" 5 hours ago Up 44 minutes telemetry e27e5086141b docker-snmp-sv2:latest "/usr/bin/supervisord" 5 hours ago Up 44 minutes snmp d63e7415c56a docker-lldp-sv2:latest "/usr/bin/docker-lld…" 5 hours ago Up About an hour lldp 96a4a65bfc34 docker-router-advertiser:latest "/usr/bin/docker-ini…" 5 hours ago Up About an hour radv 0f07e9f6aadc docker-sonic-mgmt-framework:latest "/usr/bin/supervisord" 5 hours ago Up About an hour mgmt-framework 955fdf680ddf docker-dhcp-relay:latest "/usr/bin/docker_ini…" 5 hours ago Up About an hour dhcp_relay 797ba772beec docker-syncd-brcm:latest "/usr/bin/supervisord" 5 hours ago Up About an hour syncd b0387b9a1d2b docker-teamd:latest "/usr/bin/supervisord" 5 hours ago Up About an hour teamd d2f1ecd5f4a3 docker-orchagent:latest "/usr/bin/docker-ini…" 5 hours ago Up About an hour swss c9076d2c96da docker-fpm-frr:latest "/usr/bin/supervisord" 5 hours ago Up About an hour bgp 28cb8500068a docker-platform-monitor:latest "/usr/bin/docker_ini…" 5 hours ago Up About an hour pmon 3195be5b90ce docker-database:latest "/usr/local/bin/dock…" 5 hours ago Up About an hour database root@sonic:~#

root@sonic:~# docker start nat Error response from daemon: No such container: nat Error: failed to start containers: nat

root@sonic:~# ps -ax | grep nat 8161 ttyS0 S+ 0:00 grep nat

root@sonic:~# systemctl start nat Failed to start nat.service: Unit nat.service is masked. root@sonic:~#

root@sonic:~# show ver

SONiC Software Version: SONiC.HEAD.170-c243b8a9 Distribution: Debian 9.13 Kernel: 4.9.0-11-2-amd64 Build commit: c243b8a9 Build date: Tue Aug 25 18:59:41 UTC 2020 Built by: johnar@jenkins-worker-7

Platform: x86_64-dellemc_z9264f_c3538-r0 HwSKU: DellEMC-Z9264f-C64 ASIC: broadcom Serial Number: CN030K8MDND0093E0001 Uptime: 09:04:28 up 51 min, 1 user, load average: 0.81, 0.80, 0.81

Working image: =============

root@sonic:~# ps -ax | grep nat 2797 ? Ss 0:00 /bin/bash /usr/bin/nat.sh wait 2803 ? Sl 0:04 docker wait nat 3054 ? S 0:00 python /usr/bin/supervisor-proc-exit-listener --container-name nat 3170 ? Sl 0:02 /usr/bin/natmgrd 3234 ? Sl 0:57 /usr/bin/natsyncd 7770 ttyS0 S+ 0:00 grep nat

root@sonic:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3d33718ffe73 docker-lldp-sv2:latest "/usr/bin/docker-lld…" 8 days ago Up 25 hours lldp 05f559c63a56 docker-sonic-telemetry:latest "/usr/bin/supervisord" 12 days ago Up 25 hours telemetry b28ebb2967ca docker-snmp-sv2:latest "/usr/bin/supervisord" 12 days ago Up 25 hours snmp ac912fc0a3cc docker-nat:latest "/usr/bin/supervisord" 12 days ago Up 25 hours nat >>>>>>>>>>>>>>>>>>>>>> 9394019bfed8 docker-dhcp-relay:latest "/usr/bin/docker_ini…" 12 days ago Up 25 hours dhcp_relay c68a0a4efef4 docker-router-advertiser:latest "/usr/bin/supervisord" 12 days ago Up 25 hours radv 7db3c728f766 docker-sonic-mgmt-framework:latest "/usr/bin/supervisord" 12 days ago Up 25 hours mgmt-framework 3b1703a37909 docker-syncd-brcm:latest "/usr/bin/supervisord" 12 days ago Up 25 hours syncd b7288f2f38ca docker-teamd:latest "/usr/bin/supervisord" 12 days ago Up 25 hours teamd 47707fffc9f6 docker-orchagent:latest "/usr/bin/supervisord" 12 days ago Up 25 hours swss 37560538e181 docker-fpm-frr:latest "/usr/bin/supervisord" 12 days ago Up 25 hours bgp 928ba16bfeb0 docker-platform-monitor:latest "/usr/bin/docker_ini…" 12 days ago Up 25 hours pmon 9a9517cac12b docker-database:latest "/usr/local/bin/dock…" 12 days ago Up 25 hours database root@sonic:~#

Thanks Mini

abdosi commented 4 years ago

NAT feature is disable by default. Enable it using command: sudo config feature status nat enable

mini-nair-dell commented 4 years ago

Hi,

I tried to rnable it manually, and see that, it still doesnt run. If you see in the 'working image' output, we can see that it runs, when the image is loaded itself.

root@sonic:~# sudo config feature state nat enabled

root@sonic:~# tail -f /var/log/syslog

Sep 2 06:00:06.282852 sonic INFO hostcfgd: Running cmd: 'sudo systemctl start nat.timer' Sep 2 06:00:06.296850 sonic INFO hostcfgd[1452]: Failed to start nat.timer: Unit nat.timer not found. Sep 2 06:00:06.298472 sonic ERR hostcfgd: 'sudo systemctl start nat.timer' failed. RC: 5, output: None Sep 2 06:00:06.298930 sonic INFO hostcfgd: Feature 'nat.timer' is enabled and started Sep 2 06:00:12.638989 sonic WARNING snmp#snmp-subagent [sonic_ax_impl] WARNING: Invalid mgmt IP 240.127.1.1,fe80::3e2c:30ff:fe2a:1f00 Sep 2 06:00:12.639152 sonic WARNING snmp#snmp-subagent [sonic_ax_impl] WARNING: Invalid management IP 240.127.1.1,fe80::3e2c:30ff:fe2a:1f00 Sep 2 06:00:12.642656 sonic WARNING snmp#snmp-subagent [sonic_ax_impl] WARNING: Invalid mgmt IP 240.127.1.1,fe80::3e2c:30ff:fe2a:1f00 Sep 2 06:00:12.642786 sonic WARNING snmp#snmp-subagent [sonic_ax_impl] WARNING: Invalid management IP 240.127.1.1,fe80::3e2c:30ff:fe2a:1f00 Sep 2 06:00:12.643150 sonic WARNING snmp#snmp-subagent [sonic_ax_impl] WARNING: Invalid mgmt IP 240.127.1.1,fe80::3e2c:30ff:fe2a:1f00 Sep 2 06:00:12.643310 sonic WARNING snmp#snmp-subagent [sonic_ax_impl] WARNING: Invalid management IP 240.127.1.1,fe80::3e2c:30ff:fe2a:1f00 Sep 2 06:00:21.694475 sonic NOTICE swss#orchagent: :- doTask: Get port state change notification id:1000000000010 status:2 Sep 2 06:00:21.694475 sonic NOTICE swss#orchagent: :- updatePortOperStatus: Port Ethernet132 oper state set from up to down Sep 2 06:00:21.695212 sonic NOTICE swss#orchagent: :- setHostIntfsOperStatus: Set operation status DOWN to host interface Ethernet132 Sep 2 06:00:22.046945 sonic NOTICE swss#orchagent: :- doTask: Get port state change notification id:1000000000010 status:1 Sep 2 06:00:22.046945 sonic NOTICE swss#orchagent: :- updatePortOperStatus: Port Ethernet132 oper state set from down to up Sep 2 06:00:22.047561 sonic NOTICE swss#orchagent: :- setHostIntfsOperStatus: Set operation status UP to host interface Ethernet132 ^C root@sonic:~# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 28bc2fb500f3 docker-sonic-telemetry:latest "/usr/bin/supervisord" 6 days ago Up 2 days telemetry e27e5086141b docker-snmp-sv2:latest "/usr/bin/supervisord" 6 days ago Up 2 days snmp 96a4a65bfc34 docker-router-advertiser:latest "/usr/bin/docker-ini…" 6 days ago Up 2 days radv 0f07e9f6aadc docker-sonic-mgmt-framework:latest "/usr/bin/supervisord" 6 days ago Up 2 days mgmt-framework 955fdf680ddf docker-dhcp-relay:latest "/usr/bin/docker_ini…" 6 days ago Up 2 days dhcp_relay 797ba772beec docker-syncd-brcm:latest "/usr/bin/supervisord" 6 days ago Up 2 days syncd b0387b9a1d2b docker-teamd:latest "/usr/bin/supervisord" 6 days ago Up 2 days teamd d2f1ecd5f4a3 docker-orchagent:latest "/usr/bin/docker-ini…" 6 days ago Up 2 days swss c9076d2c96da docker-fpm-frr:latest "/usr/bin/supervisord" 6 days ago Up 2 days bgp 28cb8500068a docker-platform-monitor:latest "/usr/bin/docker_ini…" 6 days ago Up 2 days pmon 3195be5b90ce docker-database:latest "/usr/local/bin/dock…" 6 days ago Up 2 days database

root@sonic:~# ps -ax | grep nat 27650 ttyS0 S+ 0:00 grep nat

root@sonic:~# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 28bc2fb500f3 docker-sonic-telemetry:latest "/usr/bin/supervisord" 6 days ago Up 2 days telemetry e27e5086141b docker-snmp-sv2:latest "/usr/bin/supervisord" 6 days ago Up 2 days snmp 96a4a65bfc34 docker-router-advertiser:latest "/usr/bin/docker-ini…" 6 days ago Up 2 days radv 0f07e9f6aadc docker-sonic-mgmt-framework:latest "/usr/bin/supervisord" 6 days ago Up 2 days mgmt-framework 955fdf680ddf docker-dhcp-relay:latest "/usr/bin/docker_ini…" 6 days ago Up 2 days dhcp_relay 797ba772beec docker-syncd-brcm:latest "/usr/bin/supervisord" 6 days ago Up 2 days syncd b0387b9a1d2b docker-teamd:latest "/usr/bin/supervisord" 6 days ago Up 2 days teamd d2f1ecd5f4a3 docker-orchagent:latest "/usr/bin/docker-ini…" 6 days ago Up 2 days swss c9076d2c96da docker-fpm-frr:latest "/usr/bin/supervisord" 6 days ago Up 2 days bgp 28cb8500068a docker-platform-monitor:latest "/usr/bin/docker_ini…" 6 days ago Up 2 days pmon 3195be5b90ce docker-database:latest "/usr/local/bin/dock…" 6 days ago Up 2 days database root@sonic:~#

Thanks Mini

mini-nair-dell commented 3 years ago

sonic_dump_sonic_20201202_200638.tar (2).gz