search

cms-sw / cmssw

CMS Offline Software
http://cms-sw.github.io/
Apache License 2.0
1.09k stars 4.33k forks source link

Build CMSSW_12_4_0_pre2 #37403

Closed perrotta closed 2 years ago

cmsbuild commented 2 years ago

Request received. I will start to build the release after one of the following approve the issue: @perrotta, @smuzaffar, @qliphy, @dpiparo. You can do this by writing "+1" in a comment. You can also ask me to begin to build cmssw-tool-conf first ( Cannot be done for patch releases ). To do this write "build cmssw-tool-conf" in a comment. I will start to build cmssw-tool-conf and then wait for the "+1" to start the build of the release. CMSSW Branch: CMSSW_12_4_X(master) Architecture: slc7_amd64_gcc10(Production), slc7_amd64_gcc11, slc7_ppc64le_gcc11, slc7_aarch64_gcc11, cs8_ppc64le_gcc11, cs8_aarch64_gcc11, cs8_amd64_gcc10, alma8_amd64_gcc10

perrotta commented 2 years ago

+1

cmsbuild commented 2 years ago

Release created: https://github.com/cms-sw/cmssw/releases/tag/CMSSW_12_4_0_pre2. The tag was created on top of branch: https://github.com/cms-sw/cmssw/tree/master

cmsbuild commented 2 years ago

Queuing Jenkins build for the following architectures: cs8_amd64_gcc10, cs8_aarch64_gcc11, slc7_ppc64le_gcc11, slc7_amd64_gcc11, slc7_amd64_gcc10, cs8_ppc64le_gcc11, slc7_aarch64_gcc11 You can abort the build by writing "Abort" in a comment. I will delete the release, the cmssw and cmsdist tag, and close the issue. You can't abort the upload once at least one achitecture is being uploaded. If you are building cmssw-tool-conf first, I will wait for each architecture to finish to start the build of cmssw.

cmsbuild commented 2 years ago

The build has started for slc7_amd64_gcc11 in cmsbuild09. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3408/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has started for slc7_amd64_gcc10 in cmsbuild08. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3407/console CMSDIST Branch: IB/CMSSW_12_4_X/master, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has started for cs8_amd64_gcc10 in cmsbuild05. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3406/console CMSDIST Branch: IB/CMSSW_12_4_X/master, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The was an error for slc7_amd64_gcc11. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_amd64_gcc11/3408

cmsbuild commented 2 years ago

The was an error for slc7_amd64_gcc10. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/3407

cmsbuild commented 2 years ago

The was an error for cs8_amd64_gcc10. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-cs8_amd64_gcc10/3406

cmsbuild commented 2 years ago

The build has started for slc7_ppc64le_gcc11 in ibmminsky-1a. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3410/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The was an error for slc7_ppc64le_gcc11. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_ppc64le_gcc11/3410

cmsbuild commented 2 years ago

The build has started for cs8_ppc64le_gcc11 in ibmminsky-1b. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3409/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The was an error for cs8_ppc64le_gcc11. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-cs8_ppc64le_gcc11/3409

cmsbuild commented 2 years ago

The build has started for cs8_aarch64_gcc11 in olarm-202a. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3411/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The was an error for cs8_aarch64_gcc11. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-cs8_aarch64_gcc11/3411

cmsbuild commented 2 years ago

The build has started for slc7_aarch64_gcc11 in olarm-202a. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3412/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The was an error for slc7_aarch64_gcc11. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_aarch64_gcc11/3412

smuzaffar commented 2 years ago

@perrotta , I am looking in to these errors

iarspider commented 2 years ago

@smuzaffar should be fixed now

iarspider commented 2 years ago

build cmssw-tool-conf

cmsbuild commented 2 years ago

The build has started for slc7_amd64_gcc10 in cmsbuild05. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3413/console CMSDIST Branch: IB/CMSSW_12_4_X/master, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has started for slc7_ppc64le_gcc11 in ibmminsky-2a. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3414/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has started for slc7_amd64_gcc11 in cmsbuild06. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3415/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has started for cs8_ppc64le_gcc11 in ibmminsky-2b. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3416/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has started for alma8_amd64_gcc10 in cmsbuild10. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3417/console CMSDIST Branch: IB/CMSSW_12_4_X/master, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The was an error for alma8_amd64_gcc10. You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-alma8_amd64_gcc10/3417

cmsbuild commented 2 years ago

The build has started for cs8_amd64_gcc10 in cmsbuild10. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3418/console CMSDIST Branch: IB/CMSSW_12_4_X/master, PKGTOOLS Branch: V00-34-XX

smuzaffar commented 2 years ago

@smuzaffar should be fixed now

thanks @iarspider , yes there was a typo in release-build script. In such cases, we need to retry the jenkins job to restart the build. The bot build command here does not work. I have restarted all the build-release jobs now

smuzaffar commented 2 years ago

alma8 build failed, PKGTOOLS/scripts/prepare-cmsdist needs fixes, I am taking care of it

cmsbuild commented 2 years ago

The build has started for alma8_amd64_gcc10 in cmsbuild21. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3419/console CMSDIST Branch: IB/CMSSW_12_4_X/master, PKGTOOLS Branch: V00-34-XX

smuzaffar commented 2 years ago

alma8 issue fixed, we had a arch name check in PKGTOOLS/scripts/prepare-cmsdist which was not recognizing alma8_amd64 as valid arch. This is fixed now and alma release is also in build process

cmsbuild commented 2 years ago

The build has started for slc7_aarch64_gcc11 in olarm-202a. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3420/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has started for cs8_aarch64_gcc11 in olarm-202b. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3421/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

cmsbuild commented 2 years ago

The build has finished sucessfully for the architecture cs8_ppc64le_gcc11 and is ready to be uploaded. You can start the uploads by writing the comment: "upload all". I will upload all the architectures as soon as the build finishes successfully. You can see the log for the build here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-cs8_ppc64le_gcc11/3416 Some tests ( runTheMatrix.py -s ) are being run, the results will be posted when done.

cmsbuild commented 2 years ago

The build has finished sucessfully for the architecture slc7_ppc64le_gcc11 and is ready to be uploaded. You can start the uploads by writing the comment: "upload all". I will upload all the architectures as soon as the build finishes successfully. You can see the log for the build here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_ppc64le_gcc11/3414 Some tests ( runTheMatrix.py -s ) are being run, the results will be posted when done.

cmsbuild commented 2 years ago

The tests have finished for cs8_ppc64le_gcc11 You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-cs8_ppc64le_gcc11/3416

Tests results: 47 46 45 34 17 4 1 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 0 failed

cmsbuild commented 2 years ago

The tests have finished for slc7_ppc64le_gcc11 You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_ppc64le_gcc11/3414

Tests results: 47 46 45 34 17 4 1 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 0 failed

cmsbuild commented 2 years ago

The build has finished sucessfully for the architecture slc7_amd64_gcc10 and is ready to be uploaded. You can start the uploads by writing the comment: "upload all". I will upload all the architectures as soon as the build finishes successfully. You can see the log for the build here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/3413 Some tests ( runTheMatrix.py -s ) are being run, the results will be posted when done.

cmsbuild commented 2 years ago

The build has finished sucessfully for the architecture cs8_amd64_gcc10 and is ready to be uploaded. You can start the uploads by writing the comment: "upload all". I will upload all the architectures as soon as the build finishes successfully. You can see the log for the build here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-cs8_amd64_gcc10/3418 Some tests ( runTheMatrix.py -s ) are being run, the results will be posted when done.

cmsbuild commented 2 years ago

The build has finished sucessfully for the architecture alma8_amd64_gcc10 and is ready to be uploaded. You can start the uploads by writing the comment: "upload all". I will upload all the architectures as soon as the build finishes successfully. You can see the log for the build here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-alma8_amd64_gcc10/3419 Some tests ( runTheMatrix.py -s ) are being run, the results will be posted when done.

cmsbuild commented 2 years ago

The build has finished sucessfully for the architecture slc7_amd64_gcc11 and is ready to be uploaded. You can start the uploads by writing the comment: "upload all". I will upload all the architectures as soon as the build finishes successfully. You can see the log for the build here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_amd64_gcc11/3415 Some tests ( runTheMatrix.py -s ) are being run, the results will be posted when done.

cmsbuild commented 2 years ago

The tests have finished for slc7_amd64_gcc10 You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/3413

Tests results: 47 46 45 34 17 4 1 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 0 failed

cmsbuild commented 2 years ago

The tests have finished for cs8_amd64_gcc10 You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-cs8_amd64_gcc10/3418

Tests results: 47 46 45 34 17 4 1 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 0 failed

cmsbuild commented 2 years ago

The tests have finished for alma8_amd64_gcc10 You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-alma8_amd64_gcc10/3419

Tests results: 47 46 45 34 17 4 1 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 0 failed

cmsbuild commented 2 years ago

The tests have finished for slc7_amd64_gcc11 You can see the log here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/auto-build-release/CMSSW_12_4_0_pre2-slc7_amd64_gcc11/3415

Tests results: 47 46 45 34 17 4 1 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 0 failed

cmsbuild commented 2 years ago

The build has started for slc7_aarch64_gcc11 in olarm-202a. You can see the progress here: https://cmssdt.cern.ch/jenkins/job/build-release/3423/console CMSDIST Branch: IB/CMSSW_12_4_X/g11, PKGTOOLS Branch: V00-34-XX

smuzaffar commented 2 years ago

I have restarted build-release job for slc7_aarch64_gcc11 which failed last night due to no space left on disk. There is cms user who has over 50GB of data on this machine ( under /data/dsouthwi ). I have asked the user to clean this up.

perrotta commented 2 years ago

I have restarted build-release job for slc7_aarch64_gcc11 which failed last night due to no space left on disk. There is cms user who has over 50GB of data on this machine ( under /data/dsouthwi ). I have asked the user to clean this up.

Thank you Shahzad. Do I understand correctly that until David cleans that space out there will be no hope of having that arch built?

smuzaffar commented 2 years ago

Not really, we still have 65GB disk free which should be enough to build the release. Last night this 65GB was not enough as we were building nearly all the externals (as we have updated pip3/setuptools). But as most of the externals were built and uploaded during the last run so existing free space should be enough to build the release.