search

adoptium / temurin-build

Eclipse Temurin™ build scripts - common across all releases/versions
Apache License 2.0
1.03k stars 249 forks source link

Ensure all builds can run on multiple machines #1044

Open sxa opened 5 years ago

sxa commented 5 years ago

This may be a duplicate, but we have some jobs that are currently locked onto specific build machines. For redundancy reasons this is a bad idea. Reasons are likely historic and hopefully not valid any more but we need to validate this and modify the tags:

EDIT 22/4/2020: The list below is now far smaller so of the original list we only have Corretto JDK8 explicitly locked to a hostname now (although others are locked for other tagging reasons - see comments)

./pipelines/jobs/configurations/jdk9_pipeline_config.groovy:                additionalNodeLabels: 'build-macstadium-macos1010-1',
./pipelines/jobs/configurations/jdk8_pipeline_config.groovy:                        corretto: 'build-macstadium-macos1010-1',
./pipelines/jobs/configurations/jdk10_pipeline_config.groovy:                additionalNodeLabels: 'build-macstadium-macos1010-1',

Original list for historic reference:

[sxa@sxat470p pipelines]$ find . -type f -print | xargs grep build- | grep groovy:
./build/openjdk11_pipeline.groovy:                        hotspot: 'build-macstadium-macos1010-1',
./build/openjdk11_pipeline.groovy:                        openj9:  'build-macstadium-macos1010-2'
./build/openjdk11_pipeline.groovy:                additionalNodeLabels : 'build-macstadium-macos1010-2',
./build/prTester/pr_test_pipeline.groovy:                    BUILD_FOLDER        : "build-scripts-pr-tester/build-test",
./build/openjdk8_pipeline.groovy:                        hotspot : 'build-macstadium-macos1010-1',
./build/openjdk8_pipeline.groovy:                        corretto: 'build-macstadium-macos1010-1',
./build/openjdk8_pipeline.groovy:                        openj9  : 'build-macstadium-macos1010-2'
./build/openjdk8_pipeline.groovy:                additionalNodeLabels : 'build-macstadium-macos1010-2',
./build/openjdk8_pipeline.groovy:                        openj9  : 'build-joyent-centos69-x64-1'
./build/openjdk_pipeline.groovy:                additionalNodeLabels: 'build-macstadium-macos1010-1',
./build/common/openjdk_build_pipeline.groovy: *  1. Check out and build JDK by calling build-farm/make-adopt-build-farm.sh
./build/common/openjdk_build_pipeline.groovy:                    def signJob = context.build job: "build-scripts/release/sign_build",
./build/common/openjdk_build_pipeline.groovy:                            projectName: "build-scripts/release/sign_build",
./build/common/openjdk_build_pipeline.groovy:        def installerJob = context.build job: "build-scripts/release/create_installer_mac",
./build/common/openjdk_build_pipeline.groovy:                projectName: "build-scripts/release/create_installer_mac",
./build/common/openjdk_build_pipeline.groovy:        def installerJob = context.build job: "build-scripts/release/create_installer_windows",
./build/common/openjdk_build_pipeline.groovy:                projectName: "build-scripts/release/create_installer_windows",
./build/common/openjdk_build_pipeline.groovy:                                    context.sh(script: "./build-farm/make-adopt-build-farm.sh")
./build/common/build_base_file.groovy:                context.build job: 'build-scripts/release/refactor_openjdk_release_tool',
./build/openjdk9_pipeline.groovy:                additionalNodeLabels: 'build-macstadium-macos1010-1',
./build/openjdk10_pipeline.groovy:                additionalNodeLabels: 'build-macstadium-macos1010-1',
[sxa@sxat470p pipelines]$

jdk8u-windows-x64-openj9 is also locked to build-softlayer-win2012r2-x64-1 for some reason at present - likely because we've historically had custom setups on the windows boxes due to the lack of a standard playbook and have installed some custom compilers and appropriate tags on certain machines

lumpfish commented 5 years ago

Also, https://ci.adoptopenjdk.net/job/build-scripts/job/release/job/create_installer_windows/ appears to be tied to a machine (maybe wix needs to be on more machines?):

(pending—Waiting for next available executor on ‘build-softlayer-win2012r2-x64-1’)
sxa commented 4 years ago

Docker build jobs also appear machine-specific

sxa commented 4 years ago
sxa commented 4 years ago

macos builds (Checked JDK11/J9) are tied to macos10.14&&build&&mac&&x64 which is only served by build-macstaduim-macos1014-1

sxa commented 4 years ago

JDK8/J9/Win32 is locked to build-softlayer-win2012r2-x64-1

sxa commented 4 years ago

All AIX/J9 builds currently locked to https://ci.adoptopenjdk.net/computer/test-ibm-aix71-ppc64-1 (Requires AIX 7.1.5)

sxa commented 4 years ago

JMC builds seem tied to the build-scalway-ubuntu system as demonstrated by https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/1336

sxa commented 4 years ago

Installers rely on https://ci.adoptopenjdk.net/label/ubuntu&&fpm/ (scaleway/x64 only)

sxa commented 4 years ago

OpenJ9 JDK8 builds now released as they can run on either of the new windows ibmcloud systems

sxa commented 3 years ago

NOTES:

M-Davies commented 3 years ago

@sxa Can this be moved to https://github.com/AdoptOpenJDK/ci-jenkins-pipelines?

sxa commented 3 years ago

@sxa Can this be moved to https://github.com/AdoptOpenJDK/ci-jenkins-pipelines?

Debatable - it's not purely a fix to the tags in there, it's also about making sure appropariate systems are available. I don't care too much where it is though.

sxa commented 3 years ago

Related: https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/1948 At the moment, only Azure systems are tagged for WiX (installer) jobs.

sxa commented 3 years ago

Note: The OpenJ9 builds (At least JDK8) are restricted to these two machines (not labelled to run on azure unlike others)

sxa commented 3 years ago

Alibaba aarch64 builds are currently locked onto one machine (they have to run on alibaba hardware for now - we'll almost certainly have to allow both machines to be options for building. Today it got locked out because the docker_build_multiarch job got scheduled on there.

sxa commented 3 years ago

SUMMARY

Many of these have now been resolved. From the top:

Remaining: