search

sonic-net / SONiC

Landing page for Software for Open Networking in the Cloud (SONiC) - https://sonic-net.github.io/SONiC/
2.29k stars 1.14k forks source link

Failed to build virtual switch with SONIC_BUILD_JOBS=4 #1738

Open dnsflood opened 5 months ago

dnsflood commented 5 months ago

Hi，

When SONIC_BUILD_JOBS is set to 4, make all results in an error, whereas the build proceeds normally when SONIC_BUILD_JOBS is set to 1. Is this a bug?

git checkout master

make init
make configure PLATFORM=vs
make SONIC_BUILD_JOBS=4 all

Below is the error log：

[ FAIL LOG START ] [ target/sonic-vs.binvsrfs.squashfs ] Build start time: Fri Jul 5 12:25:11 UTC 2024 ++ '[' -f .arch ']' ++ cat .arch

mrafatpanah commented 4 months ago

Hello. I had the same problem. I also don't know if it is a bug. I built it with SONIC_BUILD_JOBS=4 till this error happened and then I didn't use SONIC_BUILD_JOBS and built with its default mode.

dnsflood commented 4 months ago

Hello. I had the same problem. I also don't know if it is a bug. I built it with SONIC_BUILD_JOBS=4 till this error happened and then I didn't use SONIC_BUILD_JOBS and built with its default mode.

Hi, I think this might not be considered a bug. The issue arises when multiple final targets are being built in parallel, as these concurrent build tasks use some of the same files, including the root file system, scripts, etc. The solution is to run the build tasks separately for each final target, and you can still use the SONIC_BUILD_JOBS=x option.

ahsan-kaukab commented 1 month ago

Hello I am getting below error

` Build Configuration "CONFIGURED_PLATFORM" : "vs" "CONFIGURED_ARCH" : "amd64" "SONIC_CONFIG_PRINT_DEPENDENCIES" : "" "SONIC_BUILD_JOBS" : "1" "SONIC_CONFIG_MAKE_JOBS" : "8" "USE_NATIVE_DOCKERD_FOR_BUILD" : "" "SONIC_USE_DOCKER_BUILDKIT" : "" "USERNAME" : "ahsan" "PASSWORD" : "YourPaSsWoRd" "CHANGE_DEFAULT_PASSWORD" : "n" "SECURE_UPGRADE_MODE" : "no_sign" "SECURE_UPGRADE_DEV_SIGNING_KEY" : "" "SECURE_UPGRADE_SIGNING_CERT" : "" "SECURE_UPGRADE_PROD_SIGNING_TOOL": "" "SECURE_UPGRADE_PROD_TOOL_ARGS" : "" "ONIE_IMAGE_PART_SIZE" : "" "SHUTDOWN_BGP_ON_START" : "" "ENABLE_PFCWD_ON_START" : "" "SONIC_BUFFER_MODEL" : "" "INSTALL_DEBUG_TOOLS" : "" "ROUTING_STACK" : "frr" "FRR_USER_UID" : "300" "FRR_USER_GID" : "300" "ENABLE_SYNCD_RPC" : "" "SAITHRIFT_V2" : "" "ENABLE_ORGANIZATION_EXTENSIONS" : "y" "HTTP_PROXY" : "" "HTTPS_PROXY" : "" "NO_PROXY" : "" "ENABLE_ZTP" : "" "INCLUDE_PDE" : "" "SONIC_DEBUGGING_ON" : "" "SONIC_PROFILING_ON" : "" "KERNEL_PROCURE_METHOD" : "build" "BUILD_TIMESTAMP" : "20241011.153004" "BUILD_LOG_TIMESTAMP" : "none" "SONIC_IMAGE_VERSION" : "master.0-074caea" "BLDENV" : "bookworm" "VS_PREPARE_MEM" : "yes" "INCLUDE_MGMT_FRAMEWORK" : "y" "INCLUDE_ICCPD" : "n" "INCLUDE_SYSTEM_TELEMETRY" : "n" "INCLUDE_SYSTEM_GNMI" : "y" "INCLUDE_SYSTEM_EVENTD" : "y" "ENABLE_HOST_SERVICE_ON_START" : "y" "INCLUDE_RESTAPI" : "n" "INCLUDE_SFLOW" : "y" "INCLUDE_NAT" : "y" "INCLUDE_DHCP_RELAY" : "y" "INCLUDE_DHCP_SERVER" : "n" "INCLUDE_P4RT" : "n" "INCLUDE_KUBERNETES" : "n" "INCLUDE_KUBERNETES_MASTER" : "n" "INCLUDE_MACSEC" : "y" "INCLUDE_MUX" : "y" "INCLUDE_TEAMD" : "y" "INCLUDE_ROUTER_ADVERTISER" : "y" "INCLUDE_BOOTCHART : "y" "ENABLE_BOOTCHART : "n" "INCLUDE_FIPS" : "y" "ENABLE_TRANSLIB_WRITE" : "" "ENABLE_NATIVE_WRITE" : "y" "ENABLE_DIALOUT" : "" "ENABLE_AUTO_TECH_SUPPORT" : "y" "PDDF_SUPPORT" : "y" "MULTIARCH_QEMU_ENVIRON" : "n" "SONIC_VERSION_CONTROL_COMPONENTS": "none" "ENABLE_ASAN" : "n" "DEFAULT_CONTAINER_REGISTRY" : "" "BUILD_MULTIASIC_KVM" : "n" "CROSS_BUILD_ENVIRON" : "n" "LEGACY_SONIC_MGMT_DOCKER" : "y" "INCLUDE_EXTERNAL_PATCHES" : "n" "PTF_ENV_PY_VER" : "mixed"

mv: cannot move '/etc/apt/sources.list.d/debian.sources' to '/etc/apt/sources.list.d/debian.sources.back': Permission denied "SONIC_DPKG_CACHE_METHOD" : "none"

[ FAIL LOG START ] [ target/debs/bookworm/frr_8.5.4-sonic-0_amd64.deb ] Build start time: Fri Oct 11 10:30:14 UTC 2024 [ REASON ] : target/debs/bookworm/frr_8.5.4-sonic-0_amd64.deb does not exist NON-EXISTENT PREREQUISITES: target/debs/bookworm/libsnmp-dev_5.9.3+dfsg-2_amd64.deb-install target/debs/bookworm/libyang2_2.0.112-6_amd64.deb-install target/debs/bookworm/libyang2-dev_2.0.112-6_amd64.deb-install [ FLAGS FILE ] : [] [ FLAGS DEPENDS ] : [] [ FLAGS DIFF ] : [] make[1]: Entering directory '/sonic/src/sonic-frr'

pushd ./frr git checkout -b frr/8.5 origin/frr/8.5 || git checkout frr/8.5 stg branch --create stg_temp.20241011.103014 frr-8.5.4 stg import -s ../patch/series gbp dch --ignore-branch --new-version=8.5.4-sonic-0 --dch-opt="--force-bad-version" --commit --git-author cp ../dplane_fpm_sonic/dplane_fpm_sonic.c zebra/ dpkg-buildpackage -rfakeroot -b -us -uc -Ppkg.frr.nortrlib -j8 --admindir /sonic/dpkg/tmp.jAGhDXD4df stg undo || true git clean -xfdf git checkout frr/8.5 stg branch --delete --force stg_temp.20241011.103014 git rev-parse --short HEAD | xargs git checkout git checkout master git branch -D frr/8.5 popd mv frr-pythontools_8.5.4-sonic-0_all.deb frr-dbgsym_8.5.4-sonic-0_amd64.deb frr-snmp_8.5.4-sonic-0_amd64.deb frr-snmp-dbgsym_8.5.4-sonic-0_amd64.deb frr_8.5.4-sonic-0_amd64.deb /sonic/target/debs/bookworm/ /sonic/src/sonic-frr/frr /sonic/src/sonic-frr fatal: 'origin/frr/8.5' is not a commit and a branch 'frr/8.5' cannot be created from it error: pathspec 'frr/8.5' did not match any file(s) known to git make[1]: [Makefile:15: /sonic/target/debs/bookworm/frr_8.5.4-sonic-0_amd64.deb] Error 1 make[1]: Leaving directory '/sonic/src/sonic-frr' [ FAIL LOG END ] [ target/debs/bookworm/frr_8.5.4-sonic-0_amd64.deb ] make: [slave.mk:744: target/debs/bookworm/frr_8.5.4-sonic-0_amd64.deb] Error 1 make[1]: [Makefile.work:605: all] Error 2 make[1]: Leaving directory '/home/ahsan/Desktop/Ahsan-DreamBig/sonic-buildimage' make: [Makefile:47: all] Error 2 ahsan@Ahsan:~/Desktop/ ` I used following commands

`git checkout master

make init make configure PLATFORM=vs make all`

AlexFdv commented 1 month ago

The same issue for me, with SONIC_BUILD_JOBS=4 have "couldn't find these debs: ..." in the logs:

+ TARGET_PATH=target
+ scripts/build_debian_base_system.sh amd64 bookworm ./fsroot-vs
I: Target architecture can be executed
I: Retrieving InRelease 
I: Checking Release signature
I: Valid Release signature (key id 4D64FEC119C2029067D6E791F8D2585B8783D481)
I: Retrieving Packages 
I: Validating Packages 
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
E: Couldn't find these debs:  adduser apt base-files base-passwd bash bsdutils coreutils dash debconf debian-archive-keyring debianutils diffutils dpkg e2fsprogs findutils gpgv grep gzip hostname init-system-helpers libacl1 libapt-pkg6.0 libattr1 libaudit1 libblkid1 libbz2-1.0 libc-bin libc6 libcap-ng0 libcom-err2 libcrypt1 libdb5.3 libdebconfclient0 libext2fs2 libgcc-s1 libgmp10 libgnutls30 liblzma5 libmd0 libmount1 libpam-modules libpam-modules-bin libpam-runtime libpam0g libpcre2-8-0 libseccomp2 libselinux1 libsemanage2 libsmartcols1 libss2 libstdc++6 libsystemd0 libtinfo6 libudev1 libuuid1 libzstd1 login logsave mawk mount ncurses-base ncurses-bin passwd perl-base sed sysvinit-utils tar tzdata usr-is-merged util-linux util-linux-extra zlib1g 
[  FAIL LOG END  ] [ target/sonic-vs.bin__vs__rfs.squashfs ]
make: *** [slave.mk:1308: target/sonic-vs.bin__vs__rfs.squashfs] Error 1
make[1]: *** [Makefile.work:605: all] Error 2
make[1]: Leaving directory '/home/oleksii.fadieiev/projects/sonic-buildimage'
make: *** [Makefile:47: all] Error 2

Using just make all I have another error:

cd /sonic/src/p4lang/p4lang-p4c-1.2.4.2/build/ir && /usr/bin/cmake -P CMakeFiles/ir.dir/cmake_clean_target.cmake
cd /sonic/src/p4lang/p4lang-p4c-1.2.4.2/build/ir && /usr/bin/cmake -E cmake_link_script CMakeFiles/ir.dir/link.txt --verbose=1
/usr/bin/ar qc libir.a CMakeFiles/ir.dir/base.cpp.o CMakeFiles/ir.dir/dbprint.cpp.o "CMakeFiles/ir.dir/dbprint-expression.cpp.o" "CMakeFiles/ir.dir/dbprint-stmt.cpp.o" "CMakeFiles/ir.dir/dbprint-type.cpp.o" "CMakeFiles/ir.dir/dbprint-p4.cpp.o" CMakeFiles/ir.dir/dump.cpp.o CMakeFiles/ir.dir/expression.cpp.o CMakeFiles/ir.dir/ir.cpp.o CMakeFiles/ir.dir/irutils.cpp.o CMakeFiles/ir.dir/json_parser.cpp.o CMakeFiles/ir.dir/node.cpp.o CMakeFiles/ir.dir/pass_manager.cpp.o CMakeFiles/ir.dir/type.cpp.o CMakeFiles/ir.dir/v1.cpp.o CMakeFiles/ir.dir/visitor.cpp.o CMakeFiles/ir.dir/write_context.cpp.o
/usr/bin/ranlib libir.a
make[6]: Leaving directory '/sonic/src/p4lang/p4lang-p4c-1.2.4.2/build'
[ 22%] Built target ir
make[5]: Leaving directory '/sonic/src/p4lang/p4lang-p4c-1.2.4.2/build'
make[4]: *** [Makefile:169: all] Error 2
make[4]: Leaving directory '/sonic/src/p4lang/p4lang-p4c-1.2.4.2/build'
dh_auto_install: error: cd build && make -j8 install DESTDIR=/sonic/src/p4lang/p4lang-p4c-1.2.4.2/debian/p4lang-p4c AM_UPDATE_INFO_DIR=no "INSTALL=install --strip-program=true" DESTDIR=/sonic/src/p4lang/p4lang-p4c-1.2.4.2/debian/p4lang-p4c install returned exit code 2
make[3]: *** [debian/rules:47: override_dh_auto_install] Error 25
make[3]: Leaving directory '/sonic/src/p4lang/p4lang-p4c-1.2.4.2'
make[2]: *** [debian/rules:16: binary] Error 2
make[2]: Leaving directory '/sonic/src/p4lang/p4lang-p4c-1.2.4.2'
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
make[1]: *** [Makefile:38: /sonic/target/debs/bookworm/p4lang-p4c_1.2.4.2-2_amd64.deb] Error 2
make[1]: Leaving directory '/sonic/src/p4lang'
[  FAIL LOG END  ] [ target/debs/bookworm/p4lang-p4c_1.2.4.2-2_amd64.deb ]
make: *** [slave.mk:744: target/debs/bookworm/p4lang-p4c_1.2.4.2-2_amd64.deb] Error 1
make[1]: *** [Makefile.work:605: all] Error 2
make[1]: Leaving directory '/home/oleksii.fadieiev/projects/sonic-buildimage'
make: *** [Makefile:47: all] Error 2
AlexFdv commented 1 month ago

I have the same issue, it is a challenging to build a master branch on my environment: I had to rerun make with and without SONIC_BUILD_JOBS=4 parameter after each failure. But then fails on the next component(s).