Authenticating to GCR or Artifact Registry

[moesy]

Hi,

When changing the default Artifact and Chart registry to a Google Artifact Registry endpoint I receive a 403 error.

STDERR:
  Error: failed to authorize: failed to fetch anonymous token: unexpected status: 403 Forbidden

It seems the service that pushes and pulls from the registry is unaware of its GCP Iam service account despite running on GKE. My assumption is it should be using the kaniko service account that is generated however this does not appear to be true.

It appears this was supported on V2 (https://jenkins-x.io/docs/reference/components/docker-registry/#update-kubernetes-provider-configuration) however there aren't any similar instructions for v3 (https://jenkins-x.io/v3/admin/setup/config/repository/).

[jstrachan]

sounds like cloud IAM roles will need to be setup via workload identity for the artifact registry - it’s already done for GCR so the terraform just needs more roles

[jstrachan]

see https://github.com/jenkins-x/terraform-google-jx/blob/master/modules/cluster/serviceaccount.tf#L28

[rawlingsj]

yeah we could try and add the two admin roles to the tekton bot service account https://cloud.google.com/artifact-registry/docs/access-control#roles

[rawlingsj]

so adding to the terraform https://github.com/jenkins-x/terraform-google-jx/blob/master/modules/cluster/serviceaccount.tf

resource "google_project_iam_member" "tekton_sa_artifact__repo_admin_binding" {
  provider = google
  role     = "roles/artifactregistry.repoAdmin"
  member   = "serviceAccount:${google_service_account.tekton_sa.email}"
}
resource "google_project_iam_member" "tekton_sa_artifact_admin_binding" {
  provider = google
  role     = "roles/artifactregistry.admin"
  member   = "serviceAccount:${google_service_account.tekton_sa.email}"
}

[rawlingsj]

fwiw an alternative way to add extra roles to your own cluster to try things out is adding a module like this to your infrastructure repo

https://github.com/jenkins-x/jx3-infra/blob/9310b5bb9f3d1e6b7871ce77153f7d909635cf8d/main.tf#L44-L49

and then in that module add the extra roles https://github.com/jenkins-x/jx/issues/7890#issuecomment-878955137 , similar to this https://github.com/jenkins-x/jx3-infra/blob/master/modules/jenkinsxio/serviceaccounts.tf

then terraform plan and terraform apply those changes, that will give the GCP artifact admin roles to the kubernetes tekton service account which is used when the pipelines run so should fix the 403 above.

[moesy]

Thank you @rawlingsj and @jstrachan. I will be out of the office for a few days this week and will pick this up to test when I return.

[moesy]

It look's like it is the jx-boot container that attempts to pull from artifact registry. Is there a reason to apply these perms to the tekton SA as well?

The auth command gcloud auth configure-docker us-central1-docker.pkg.dev,asia-northeast1-docker.pkg.dev typically needs to be runned as well otherwise I think the 403 errors will remain.

https://cloud.google.com/artifact-registry/docs/docker/authentication