[Blueprints, 02-at-scale] HA Controllers: Shared lib are not loaded correctly

[carlosrodlop]

Description

Please provide a clear and concise description of the issue you are encountering, and a reproduction of your configuration. The reproduction MUST be executable by running terraform init && terraform apply without any further changes.

If your request is for a new feature, please use the Feature request template.

• [X] I have searched the open/closed issues in this repository and my issue is not listed.
• [X] I have checked that local tests are passing.
• [X] If the issue is related to an AWS EKS add-on, I have searched the open/closed issues in the upstream aws-ia/terraform-aws-eks-blueprints and my issue is not listed.
• [X] If the issue is related to an AWS EKS add-on, I have checked that upstream tests for the eks terraform blueprints add-on are passing.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version [Required]:

• Terraform version:

• Provider version(s):

Reproduction Code [Required]

I followed the the recommendations explained in Shared Lib for HA and it don’t get Shared lib loading working on HA controllers.

• Shared lib repo: https://github.com/cloudbees/terraform-aws-cloudbees-ci-eks-addon/tree/develop/blueprints/02-at-scale/cbci/shared-lib (I use a monorepo using spare checkout)
• The Shared lib is loaded into the HA controller like this configuration
• The error is show in console output like:

ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --depth=1 -- https://github.com/cloudbees/terraform-aws-cloudbees-ci-eks-addon.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: detected dubious ownership in repository at '/var/jenkins_home/jobs/admin/jobs/validations/jobs/kaniko/builds/1/libs/3c2f85b3e5660c2476ae27485be37e361d598df28f9785685ac76198ec5c0f4f/root'
To add an exception for this directory, call:

    git config --global --add safe.directory /var/jenkins_home/jobs/admin/jobs/validations/jobs/kaniko/builds/1/libs/3c2f85b3e5660c2476ae27485be37e361d598df28f9785685ac76198ec5c0f4f/root
 …
ERROR: Error cloning remote repo 'origin'
ERROR: Maximum checkout retry attempts reached, aborting
ERROR: Error cloning remote repo 'origin'
ERROR: Maximum checkout retry attempts reached, aborting
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: Loading libraries failed

• Note that the same pipeline loads correctly the Shared lib in the None HA controller configured like this

Steps to reproduce the behaviour: Deploy 02 At Scale blueprints > team-c-ha] > admin > validations and run the following pipelines than uses Shared Lib:

• kaniko
• s3-WScacheAndArtifacts

IMPORTANT: same pipelines load Shared lib correctly for None HA controllers

Expected behavior

The Shared lib are loaded correctly

Actual behavior

The Shared lib are NOT loaded correctly

Terminal Output Screenshot(s)

Additional context

[jglick]

Note that as of the November 2024 feature release (2.479.2.x I think), HA managed controllers will default to using the /tmp volume to check out Groovy libraries, and you will no longer be required to select “clone” mode.

BTW several URLs in the description are now 404s. Please use permalinks (Y shortcut) to defend against this.