Syslog error messages are observed when doing qos reload with EVPN/VxLAN configurations

[congh-nvidia]

Description

Steps to reproduce the issue:

1. Config port channels sudo config portchannel add PortChannel0001 sudo config portchannel member add PortChannel0001 Ethernet248 sudo config portchannel add PortChannel0002 sudo config portchannel member add PortChannel0002 Ethernet504
2. Config vlan sudo config vlan add 40 sudo config vlan member add 40 PortChannel0002 sudo config vlan member add 40 Ethernet0 sudo config vlan add 69 sudo config vlan member add 69 PortChannel0002 sudo config vlan add 690 sudo config vlan member add 690 Ethernet0 sudo config vlan add 10 sudo config vlan member add 10 Ethernet0 sudo config vlan add 50 sudo config vlan member add 50 Ethernet256 sudo config vlan add 100 sudo config vlan member add 100 PortChannel0002 sudo config vlan member add 100 Ethernet0 sudo config vlan add 101 sudo config vlan member add 101 PortChannel0002 sudo config vlan member add 101 Ethernet0 sudo config vlan add 20 sudo config vlan member add 20 PortChannel0002 sudo config vlan add 200
3. Config IP sudo config interface ip add Vlan40 40.0.0.1/24 sudo config interface ip add Vlan40 4000::1/64 sudo config interface ip add PortChannel0001 30.0.0.1/24 sudo config interface ip add PortChannel0001 3000::1/64 sudo config interface ip add Vlan69 69.0.0.1/24 sudo config interface ip add Vlan69 6900::1/64 sudo config interface ip add Vlan690 69.0.1.1/24 sudo config interface ip add Vlan690 6900:1::1/64 sudo config interface ip add Vlan50 50.0.0.1/24 sudo config interface ip add Vlan10 10.0.1.1/24 sudo config interface ip add Loopback0 10.1.0.32/32 sudo config interface ip add Vlan100 100.0.0.1/24 sudo config interface ip add Vlan100 100::1/64 sudo config interface ip add Vlan101 101.0.0.1/24 sudo config interface ip add Vlan101 101::1/64 sudo config interface ip add Vlan20 20.0.0.1/24
4. Config EVPN/VxLAN sudo config vxlan map add vtep101032 69 76543 sudo config vxlan add vtep101032 10.1.0.32 sudo config vxlan evpn_nvo add my-nvo vtep101032 sudo config vxlan map add vtep101032 69 76543 sudo config vxlan map add vtep101032 100 500100 sudo config vxlan map add vtep101032 101 500101 sudo config vxlan map add vtep101032 20 50020 sudo config vxlan map add vtep101032 200 500200 sudo config vrf add_vrf_vni_map Vrf1 500200
5. qos reload config qos reload
6. Check the errors in syslog

Describe the results you received:

Jul 13 05:30:59.701662 r-moose-02 ERR swss#vxlanmgrd: :- doVxlanEvpnNvoCreateTask: Only Single NVO object allowed
Jul 13 05:30:59.706036 r-moose-02 ERR swss#vrfmgrd: :- doVrfVxlanTableCreateTask:  vni 500200 is already mapped to vrf Vrf1
Jul 13 05:30:59.706054 r-moose-02 ERR swss#vrfmgrd: :- doTask: VRF VNI Map Config Failed
Jul 13 05:30:59.724347 r-moose-02 ERR swss#vxlanmgrd: :- doVxlanTunnelMapCreateTask: Map already present : vtep101032:map_500100_Vlan100
Jul 13 05:30:59.724361 r-moose-02 ERR swss#vxlanmgrd: :- doVxlanTunnelMapCreateTask: Map already present : vtep101032:map_500101_Vlan101
Jul 13 05:30:59.724361 r-moose-02 ERR swss#vxlanmgrd: :- doVxlanTunnelMapCreateTask: Map already present : vtep101032:map_500200_Vlan200
Jul 13 05:30:59.724366 r-moose-02 ERR swss#vxlanmgrd: :- doVxlanTunnelMapCreateTask: Map already present : vtep101032:map_50020_Vlan20
Jul 13 05:30:59.724371 r-moose-02 ERR swss#vxlanmgrd: :- doVxlanTunnelMapCreateTask: Map already present : vtep101032:map_76543_Vlan69
Jul 13 05:30:59.750999 r-moose-02 ERR swss#intfmgrd: :- setIntfIp: Command '/sbin/ip -6 address "add" "100::1/64" broadcast "100::ffff:ffff:ffff:ffff" dev "Vlan100"' failed with rc 2
Jul 13 05:30:59.760038 r-moose-02 ERR swss#intfmgrd: :- setIntfIp: Command '/sbin/ip -6 address "add" "101::1/64" broadcast "101::ffff:ffff:ffff:ffff" dev "Vlan101"' failed with rc 2
Jul 13 05:30:59.776595 r-moose-02 ERR swss#intfmgrd: :- setIntfIp: Command '/sbin/ip -6 address "add" "4000::1/64" broadcast "4000::ffff:ffff:ffff:ffff" dev "Vlan40"' failed with rc 2
Jul 13 05:30:59.787899 r-moose-02 ERR swss#intfmgrd: :- setIntfIp: Command '/sbin/ip -6 address "add" "6900:1::1/64" broadcast "6900:1::ffff:ffff:ffff:ffff" dev "Vlan690"' failed with rc 2
Jul 13 05:30:59.793684 r-moose-02 ERR swss#intfmgrd: :- setIntfIp: Command '/sbin/ip -6 address "add" "6900::1/64" broadcast "6900::ffff:ffff:ffff:ffff" dev "Vlan69"' failed with rc 2
Jul 13 05:30:59.796160 r-moose-02 ERR swss#intfmgrd: :- setIntfIp: Command '/sbin/ip address "add" "10.1.0.32/32" dev "Loopback0"' failed with rc 2
Jul 13 05:30:59.803859 r-moose-02 ERR swss#intfmgrd: :- setIntfIp: Command '/sbin/ip -6 address "add" "3000::1/64" broadcast "3000::ffff:ffff:ffff:ffff" dev "PortChannel0001"' failed with rc 2
Jul 13 05:31:01.069089 r-moose-02 ERR swss#orchagent: :- addOperation: Vxlan tunnel 'vtep101032' is already exists

Describe the results you expected:

Doing qos reload while there is EVPN/VxLAN configurations in config_db is a valid operation, there should not be errors in the syslog.

Output of show version:

SONiC Software Version: SONiC.202211_1_RC2.25-23bbcd5d9_Internal
SONiC OS Version: 11
Distribution: Debian 11.7
Kernel: 5.10.0-18-2-amd64
Build commit: 23bbcd5d9
Build date: Wed Jul 12 11:08:25 UTC 2023
Built by: sw-r2d2-bot@r-build-sonic-ci03-241

Platform: x86_64-nvidia_sn5600-r0
HwSKU: ACS-SN5600
ASIC: mellanox
ASIC Count: 1
Serial Number: MT2315XZ04ZQ
Model Number: 920-9N42F-00RS-5NA
Hardware Revision: A1
Uptime: 06:25:35 up  2:38,  1 user,  load average: 1.47, 1.45, 1.64
Date: Thu 13 Jul 2023 06:25:35

Docker images:
REPOSITORY                                         TAG                                  IMAGE ID       SIZE
docker-syncd-mlnx                                  202211_1_RC2.25-23bbcd5d9_Internal   18c6dda81b71   964MB
docker-syncd-mlnx                                  latest                               18c6dda81b71   964MB
docker-platform-monitor                            202211_1_RC2.25-23bbcd5d9_Internal   42dacc498c20   963MB
docker-platform-monitor                            latest                               42dacc498c20   963MB
docker-dhcp-relay                                  latest                               3f95b0989ff9   452MB
docker-macsec                                      latest                               b5ad5444d8ca   461MB
docker-eventd                                      202211_1_RC2.25-23bbcd5d9_Internal   3b752987a3e2   443MB
docker-eventd                                      latest                               3b752987a3e2   443MB
urm.nvidia.com/sw-nbu-sws-sonic-docker/sonic-wjh   1.5.3-202211-13                      14978ef516bc   432MB
docker-orchagent                                   202211_1_RC2.25-23bbcd5d9_Internal   391a1fb0f42d   475MB
docker-orchagent                                   latest                               391a1fb0f42d   475MB
docker-teamd                                       202211_1_RC2.25-23bbcd5d9_Internal   9444552cf357   456MB
docker-teamd                                       latest                               9444552cf357   456MB
docker-snmp                                        202211_1_RC2.25-23bbcd5d9_Internal   a57c03b18b12   484MB
docker-snmp                                        latest                               a57c03b18b12   484MB
docker-fpm-frr                                     202211_1_RC2.25-23bbcd5d9_Internal   3d7ac3d0e139   485MB
docker-fpm-frr                                     latest                               3d7ac3d0e139   485MB
docker-sonic-telemetry                             202211_1_RC2.25-23bbcd5d9_Internal   1436f5da47ae   737MB
docker-sonic-telemetry                             latest                               1436f5da47ae   737MB
docker-sonic-p4rt                                  202211_1_RC2.25-23bbcd5d9_Internal   91fba50cc4c1   521MB
docker-sonic-p4rt                                  latest                               91fba50cc4c1   521MB
docker-router-advertiser                           202211_1_RC2.25-23bbcd5d9_Internal   2e67e4d49612   439MB
docker-router-advertiser                           latest                               2e67e4d49612   439MB
docker-lldp                                        202211_1_RC2.25-23bbcd5d9_Internal   fa3b42549e01   481MB
docker-lldp                                        latest                               fa3b42549e01   481MB
docker-mux                                         202211_1_RC2.25-23bbcd5d9_Internal   b4533bd21c95   488MB
docker-mux                                         latest                               b4533bd21c95   488MB
docker-database                                    202211_1_RC2.25-23bbcd5d9_Internal   72a70834b3ea   439MB
docker-database                                    latest                               72a70834b3ea   439MB
docker-sonic-mgmt-framework                        202211_1_RC2.25-23bbcd5d9_Internal   43dce34a9550   552MB
docker-sonic-mgmt-framework                        latest                               43dce34a9550   552MB
docker-sflow                                       202211_1_RC2.25-23bbcd5d9_Internal   1c6b54e53b51   422MB
docker-sflow                                       latest                               1c6b54e53b51   422MB
docker-nat                                         202211_1_RC2.25-23bbcd5d9_Internal   bafaa375d923   424MB
docker-nat                                         latest                               bafaa375d923   424MB
urm.nvidia.com/sw-nbu-sws-sonic-docker/doai        1.0.0-master-internal-25             475e4a384e19   201MB
harbor.mellanox.com/sonic/cpu-report               10.0.0                               5314b41a2a5e   413MB

Output of show techsupport:

[sonic_dump_r-moose-02_20230713_062522.zip](https://github.com/sonic-net/sonic-buildimage/files/12079279/sonic_dump_r-moose-02_20230713_062522.zip)

Additional information you deem important (e.g. issue happens only occasionally):

[dgsudharsan]

Here is the attachment of techsupport https://github.com/sonic-net/sonic-buildimage/files/12079279/sonic_dump_r-moose-02_20230713_062522.zip

[dgsudharsan]

@adyeung Can you please provide an ETA for fixing this?

[dgsudharsan]

@srj102 Can you please provide an ETA for fix?