search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
701 stars 1.35k forks source link

mclagdctl missing from Broadcom build #9855

Open aussieserverhosts opened 2 years ago

aussieserverhosts commented 2 years ago

Description

mclagdctl missing from Broadcom build

Steps to reproduce the issue:

Attempting to setup MCLAG on two identical switches using this article: https://support.edge-core.com/hc/en-us/articles/900002380706--Edgecore-SONiC-MC-LAG Also tested with master branch 


SONiC Software Version: SONiC.master.67679-49a036e90
Distribution: Debian 11.2
Kernel: 5.10.0-8-2-amd64
Build commit: 49a036e90
Build date: Tue Jan 25 18:17:00 UTC 2022
Built by: AzDevOps@sonic-build-workers-0013WI

Describe the results you received:

Missing mclagdctl and other mclag related commands Unable to "show mclag"

Usage: show [OPTIONS] COMMAND [ARGS]...
Try "show -h" for help.

Error: No such command "mclag".

Describe the results you expected:

Output of show version:

show version

SONiC Software Version: SONiC.202111.67173-82e0659dd
Distribution: Debian 11.2
Kernel: 5.10.0-8-2-amd64
Build commit: 82e0659dd
Build date: Sun Jan 23 20:05:32 UTC 2022
Built by: AzDevOps@sonic-build-workers-0013P2

Platform: x86_64-cel_seastone-r0
HwSKU: Seastone-DX010
ASIC: broadcom
ASIC Count: 1
Serial Number: N/A
Model Number: N/A
Hardware Revision: N/A
Uptime: 09:48:55 up 1 day, 22:48,  3 users,  load average: 0.89, 0.48, 0.40

Docker images:
REPOSITORY                    TAG                      IMAGE ID       SIZE
docker-dhcp-relay             latest                   94d236ff71c7   436MB
docker-fpm-frr                202111.67173-82e0659dd   0b7774a4e1dd   455MB
docker-fpm-frr                latest                   0b7774a4e1dd   455MB
docker-platform-monitor       202111.67173-82e0659dd   4a0aacf24539   689MB
docker-platform-monitor       latest                   4a0aacf24539   689MB
docker-macsec                 202111.67173-82e0659dd   484a70c7f315   440MB
docker-macsec                 latest                   484a70c7f315   440MB
docker-syncd-brcm             202111.67173-82e0659dd   757f0f5e99dd   739MB
docker-syncd-brcm             latest                   757f0f5e99dd   739MB
docker-teamd                  202111.67173-82e0659dd   b68539506da6   437MB
docker-teamd                  latest                   b68539506da6   437MB
docker-snmp                   202111.67173-82e0659dd   9c4aa2d2f21e   465MB
docker-snmp                   latest                   9c4aa2d2f21e   465MB
docker-sonic-mgmt-framework   202111.67173-82e0659dd   82d8588e2c3b   578MB
docker-sonic-mgmt-framework   latest                   82d8588e2c3b   578MB
docker-sonic-telemetry        202111.67173-82e0659dd   a5bb4cbd7418   511MB
docker-sonic-telemetry        latest                   a5bb4cbd7418   511MB
docker-sflow                  202111.67173-82e0659dd   34b599ab31c9   438MB
docker-sflow                  latest                   34b599ab31c9   438MB
docker-router-advertiser      202111.67173-82e0659dd   9d6bf7442b37   423MB
docker-router-advertiser      latest                   9d6bf7442b37   423MB
docker-orchagent              202111.67173-82e0659dd   0e6ac7795d39   455MB
docker-orchagent              latest                   0e6ac7795d39   455MB
docker-nat                    202111.67173-82e0659dd   b17a3d72ec72   440MB
docker-nat                    latest                   b17a3d72ec72   440MB
docker-mux                    202111.67173-82e0659dd   8a2f1d0aff05   475MB
docker-mux                    latest                   8a2f1d0aff05   475MB
docker-lldp                   202111.67173-82e0659dd   13929fdc0791   463MB
docker-lldp                   latest                   13929fdc0791   463MB
docker-database               202111.67173-82e0659dd   225b6b90afda   423MB
docker-database               latest                   225b6b90afda   423MB
docker-gbsyncd-credo          202111.67173-82e0659dd   797bfb107f1d   477MB
docker-gbsyncd-credo          latest                   797bfb107f1d   477MB

Output of show techsupport:

Additional information you deem important (e.g. issue happens only occasionally):

shinglerb commented 2 years ago

I have encountered the same issue on the mellanox build

show version:

SONiC Software Version: SONiC.master.67675-49a036e90
Distribution: Debian 11.2
Kernel: 5.10.0-8-2-amd64
Build commit: 49a036e90
Build date: Tue Jan 25 18:54:09 UTC 2022
Built by: AzDevOps@sonic-build-workers-0013XQ

Platform: x86_64-mlnx_msn2010-r0
HwSKU: ACS-MSN2010
ASIC: mellanox
ASIC Count: 1
Serial Number: MT2115X15625
Model Number: MSN2010-CB2F
Hardware Revision: A3
Uptime: 12:04:14 up 2 days, 18:34,  1 user,  load average: 0.84, 0.74, 0.70

Docker images:
REPOSITORY                    TAG                      IMAGE ID       SIZE
docker-syncd-mlnx             latest                   189f0f3efa2e   1.01GB
docker-syncd-mlnx             master.67675-49a036e90   189f0f3efa2e   1.01GB
docker-dhcp-relay             latest                   85820d32c625   445MB
docker-sonic-telemetry        latest                   64080c96a8ec   524MB
docker-sonic-telemetry        master.67675-49a036e90   64080c96a8ec   524MB
docker-fpm-frr                latest                   a81605b6c9f5   456MB
docker-fpm-frr                master.67675-49a036e90   a81605b6c9f5   456MB
docker-sflow                  latest                   1a523977d36a   438MB
docker-sflow                  master.67675-49a036e90   1a523977d36a   438MB
docker-teamd                  latest                   f278e0dcbc06   438MB
docker-teamd                  master.67675-49a036e90   f278e0dcbc06   438MB
docker-orchagent              latest                   0679991a38b2   457MB
docker-orchagent              master.67675-49a036e90   0679991a38b2   457MB
docker-nat                    latest                   a099ba9e7ece   440MB
docker-nat                    master.67675-49a036e90   a099ba9e7ece   440MB
docker-macsec                 latest                   b105de54ff7d   440MB
docker-macsec                 master.67675-49a036e90   b105de54ff7d   440MB
docker-platform-monitor       latest                   879ed4e31632   653MB
docker-platform-monitor       master.67675-49a036e90   879ed4e31632   653MB
docker-snmp                   latest                   5767f4614529   465MB
docker-snmp                   master.67675-49a036e90   5767f4614529   465MB
docker-sonic-mgmt-framework   latest                   c41f01f59a6b   578MB
docker-sonic-mgmt-framework   master.67675-49a036e90   c41f01f59a6b   578MB
docker-router-advertiser      latest                   f64e858e2f96   423MB
docker-router-advertiser      master.67675-49a036e90   f64e858e2f96   423MB
docker-lldp                   latest                   ad59fb5553af   463MB
docker-lldp                   master.67675-49a036e90   ad59fb5553af   463MB
docker-mux                    latest                   9aae8d1ba3fd   476MB
docker-mux                    master.67675-49a036e90   9aae8d1ba3fd   476MB
docker-database               latest                   11f4ae14bed5   423MB
docker-database               master.67675-49a036e90   11f4ae14bed5   423MB
zhangyanzhao commented 2 years ago

BRCM team will take a look.

Praveen-Brcm commented 2 years ago

Hi all: Mclagdctl command is not part of the show Tree. Please consider execute the commands directly. examples below

mclagdctl dump state

mclagdctl -i dump mac

thanks

aussieserverhosts commented 2 years ago

-bash: mclagctl: command not found

Praveen-Brcm commented 2 years ago

@aussieserverhosts did you try mclagdctl or mclagctl .? Can you please check if the ICCPd is running as well .?

docker ps will show if ICCP is running.

thanks.

aussieserverhosts commented 2 years ago

@Praveen-Brcm both

ICCP is not running and isn't even on the list

docker ps --all
CONTAINER ID   IMAGE                                COMMAND                  CREATED       STATUS                   PORTS     NAMES
e45815376070   docker-snmp:latest                   "/usr/local/bin/supe…"   2 weeks ago   Up 13 days                         snmp
1f6a6ef11258   docker-sonic-telemetry:latest        "/usr/local/bin/supe…"   2 weeks ago   Up 13 days                         telemetry
386f8c67e198   docker-sonic-mgmt-framework:latest   "/usr/local/bin/supe…"   2 weeks ago   Up 13 days                         mgmt-framework
c60db1d649ff   docker-router-advertiser:latest      "/usr/bin/docker-ini…"   2 weeks ago   Up 13 days                         radv
637323d2d6f7   docker-lldp:latest                   "/usr/bin/docker-lld…"   2 weeks ago   Up 13 days                         lldp
64955a1c3807   docker-syncd-brcm:latest             "/usr/local/bin/supe…"   2 weeks ago   Up 13 days                         syncd
ed3e2f79c3f7   docker-teamd:latest                  "/usr/local/bin/supe…"   2 weeks ago   Up 13 days                         teamd
64fc904ce1cc   docker-orchagent:latest              "/usr/bin/docker-ini…"   2 weeks ago   Up 13 days                         swss
ac8456a16197   docker-fpm-frr:latest                "/usr/bin/docker_ini…"   2 weeks ago   Exited (0) 2 weeks ago             bgp
cc2cd06484db   docker-platform-monitor:latest       "/usr/bin/docker_ini…"   2 weeks ago   Up 13 days                         pmon
49293355b783   docker-database:latest               "/usr/local/bin/dock…"   2 weeks ago   Up 2 weeks                         database
Praveen-Brcm commented 2 years ago

@aussieserverhosts : Can you please check in your repo sonic-buildimage/rules/config if ICCPD is included for the build .?

INCLUDE_ICCPD - build docker-iccpd for mclag support

INCLUDE_ICCPD = y -- please make sure this is Y if N, then change and rebuild image.

Thanks

aussieserverhosts commented 2 years ago

It does not, do we need to build it ourselves? We simply pulled the latest built image from the pipelines here: https://sonic-build.azurewebsites.net/ui/sonic/pipelines/138/builds/70590/artifacts/130617?branchName=202111&artifactName=sonic-buildimage.broadcom

Praveen-Brcm commented 2 years ago

Default behavior was changed as part of this PR by lguohan https://github.com/Azure/sonic-buildimage/pull/4502 .

We would need to change the config rules and re-build. Thanks.

aussieserverhosts commented 2 years ago

Build appears to be failing based on current 202111 branch with instructions here: https://github.com/Azure/sonic-buildimage What should be build environment be? I can see that it spins up a docker container to build, but currently failing.

I've attached the log here: buildlog.log

The only changes were in rules/config including ICCPD on line 126 and Makefile to only build Bullseye

gregory-mac commented 1 year ago

Hi! Might be the wrong place to ask, but are there any plans to reinstate MLAG functionality?

sjhwilkes commented 1 year ago

Is anyone using MCLAG in production out there? When I build successfully with IPCC I seem to hit issues with production levels of traffic on MCLAGs. Currently running with 2nd ToR in cold standby as LAGs to a single chassis are rock solid as expected. (this on on Broadcom based platform)

amitlimaye commented 1 year ago

@sjhwilkes what kind of problems did you encounter running this is in production. We are trying to explore the mclag path as well.

TafkaMax commented 5 months ago

Wait but you can still configure MCLAG via sudo config mclag etc...?

I am following this tutorial and ran into this issue: https://support.edge-core.com/hc/en-us/articles/900002380706--Enterprise-SONiC-MC-LAG

So the builds downloadable from https://sonic.software/ don't have MCLAG functionality built into them? But the builds I can get from vendors such as Edgecore and Dell (Broadcom Enterprise SONiC) have MCLAG enabled?

johnnycivic66 commented 2 months ago

This is frustrating as I've been having some 'fun' with a pair of Mellanox SN2700's for a few weeks now. I was able to (sort of?) successfully build an image that installed on both of my switches, though with different outcomes as far as how many docker images appear to actually be running.

Is SONiC not a good option for MC-LAG? I have them build and am going to build/test these over the next week but I'm wondering if ponying up the money for a supported OS from NVidia is the right move?

johnnycivic66 commented 2 months ago

Wait but you can still configure MCLAG via sudo config mclag etc...?

I am following this tutorial and ran into this issue: https://support.edge-core.com/hc/en-us/articles/900002380706--Enterprise-SONiC-MC-LAG

So the builds downloadable from https://sonic.software/ don't have MCLAG functionality built into them? But the builds I can get from vendors such as Edgecore and Dell (Broadcom Enterprise SONiC) have MCLAG enabled?

If you make sure that iccpd is unmasked "sudo systemctl unmask iccpd", and then running "systemctl start iccpd" and then check that the container is running with a 'docker ps', then you should be able to get it working.

I follwed the Edgecore docs as well and have MC-LAG functioning. Just not sure about putting it into production.