Set up PAT for GCB ⇔ GHA

[chadwhitacre]

For https://github.com/getsentry/self-hosted/issues/1756.

From https://getsentry.atlassian.net/servicedesk/customer/portal/4/ISD-3978:

We decided to have Google Cloud Build continue to push to DockerHub, and instead trigger a Github workflow to run tests through an API request. To accomplish this goal we need to have a Personal Access Token for one of the bot accounts so we can make an API call to kick off the end-to-end tests workflow on Github from the Google Cloud Build runner. This token would need to be added to the 3 cloudbuild.yml files in sentry, relay, and snuba. It seems BYK left behind instructions on how he added the Docker secret here: sentry/cloudbuild.yaml at 7047bc3df8b112b97652df4fc21f7ea1ce008509 · getsentry/sentry

The addition of this secret should be secure, as it will only be used on GCB. It should only need the repo scope.

I have the bot access I need.

To Do

• [ ] Do I have the GCP access I need?
• [ ] Go through docs.
• [ ] Add encrypted secret to cloudbuild.yml:
  • [ ] relay
  • [ ] snuba
  • [ ] sentry

[chadwhitacre]

Do I have the GCP access I need?

[chadwhitacre]

I think I do. I can see a key ring called service-credentials with a key called cloudbuild. That seems to match what I find in the cloudbuild.yml:

  - kmsKeyName: projects/sentryio/locations/global/keyRings/service-credentials/cryptoKeys/cloudbuild

[chadwhitacre]

Looking at IAM, I also have "Secret Manager Admin," so I should be good? 🤞

[chadwhitacre]

I guess what's throwing me off is that the only two secrets I see in Secret Manager are named github-app-[fooid], I expect to see something like docker-password in there. 🤔

[chadwhitacre]

I mean I guess I don't necessarily need to care about where DOCKER_PASSWORD is coming from if I can get SELF_HOSTED_E2E_PAT to work. ¯\_(ツ)_/¯

[chadwhitacre]

Okay I created a new secret and now I'm looking at this step:

Grant the Secret Manager Secret Accessor (roles/secretmanager.secretAccessor) IAM role for the secret to the Cloud Build service account:

But I really don't understand why it doesn't already have this role in order for the docker password to work. 🤔

I guess the question of whether I'm in the right environment (project) boils down to, is this where Cloud Build is demonstrably running CI for sentry, relay, and snuba?

[chadwhitacre]

projects/sentryio is in kmsKeyName.

[chadwhitacre]

Yeah okay and I see everything in Cloud Build—I've been here plenty of times, just double-checking. 😅

[chadwhitacre]

The service account is already a principle for the secret.

[chadwhitacre]

Looking at encryption next.

[chadwhitacre]

Now that you have a key, you can use that key to encrypt text or binary content.

https://cloud.google.com/kms/docs/create-encryption-keys#encrypt_data

[chadwhitacre]

Looks like I have gcloud already installed. :phew:

[chadwhitacre]

#!/usr/bin/env zsh
gcloud kms encrypt \
    --location "global" \
    --project "sentryio" \
    --keyring "service-credentials" \
    --key "cloudbuild" \
    --plaintext-file pat \
    --ciphertext-file pat.enc

$ ./encrypt
API [cloudkms.googleapis.com] not enabled on project [301117755169]. Would you like to enable and retry (this
will take a few minutes)? (y/N)?  n

ERROR: (gcloud.kms.encrypt) PERMISSION_DENIED: Cloud Key Management Service (KMS) API has not been used in project 301117755169 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudkms.googleapis.com/overview?project=301117755169 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
- '@type': type.googleapis.com/google.rpc.Help
  links:
  - description: Google developers console API activation
    url: https://console.developers.google.com/apis/api/cloudkms.googleapis.com/overview?project=301117755169
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: googleapis.com
  metadata:
    consumer: projects/301117755169
    service: cloudkms.googleapis.com
  reason: SERVICE_DISABLED
$

That's not the project number of the sentryio project. 🤔

[chadwhitacre]

https://console.cloud.google.com/iam-admin/settings?project=301117755169

resolves to

https://console.cloud.google.com/iam-admin/settings?project=self-hosted-lol

Must be something in local config for gcloud ...

[chadwhitacre]

Ah, leftover from hackweek trailcam project. ☺️

[chadwhitacre]

$ gcloud config set project sentryio
Updated property [core/project].
$

[chadwhitacre]

#!/usr/bin/env zsh
gcloud kms encrypt \
    --project "sentryio" \
    --location "global" \
    --keyring "service-credentials" \
    --key "cloudbuild" \
    --plaintext-file pat \
    --ciphertext-file pat.enc

$ ./encrypt 
ERROR: (gcloud.kms.encrypt) PERMISSION_DENIED: Permission 'cloudkms.cryptoKeyVersions.useToEncrypt' denied on resource 'projects/sentryio/locations/global/keyRings/service-credentials/cryptoKeys/cloudbuild' (or it may not exist).
$

[chadwhitacre]

I have a vague memory of having to do this with the service account before, maaaaaaaybe for the docker thing though I think that was just before my time?

[chadwhitacre]

https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account

[chadwhitacre]

$ gcloud iam service-accounts keys create key.json --iam-account=sentryio@appspot.gserviceaccount.com
created key [deadbeef] of type [json] as [key.json] for [sentryio@appspot.gserviceaccount.com]
$

💃

This account has the needed permission.

[chadwhitacre]

#!/usr/bin/env zsh
set -e

gcloud config set account chadwhitacre@sentry.io
gcloud config set project sentryio

account="sentryio@appspot.gserviceaccount.com"
account="345757944225@cloudbuild.gserviceaccount.com"
gcloud iam service-accounts keys create key.json --iam-account $account
gcloud auth activate-service-account $account --key-file key.json

gcloud kms encrypt \
  --location global \
  --keyring service-credentials \
  --key cloudbuild \
  --plaintext-file pat \
  --ciphertext-file pat.enc

$ ./encrypt                                       
Updated property [core/project].
created key [foobar] of type [json] as [key.json] for [sentryio@appspot.gserviceaccount.com]
Activated service account credentials for: [sentryio@appspot.gserviceaccount.com]
ERROR: (gcloud.kms.encrypt) PERMISSION_DENIED: Permission 'cloudkms.cryptoKeyVersions.useToEncrypt' denied on resource 'projects/sentryio/locations/global/keyRings/service-credentials/cryptoKeys/cloudbuild' (or it may not exist).
$

This account appears to have "Cloud KMS CryptoKey Decrypter" for the cloudbuild key, so I'm not sure what's up. With the other account I make it even less far:

$ ./encrypt 
Updated property [core/account].
Updated property [core/project].
ERROR: (gcloud.iam.service-accounts.keys.create) NOT_FOUND: Unknown service account
$

[chadwhitacre]

Bailed.