search

kubernetes / test-infra

Test infrastructure for the Kubernetes project.
Apache License 2.0
3.83k stars 2.64k forks source link

sync credentials to k8s-infra #27861

Closed ameukam closed 1 year ago

ameukam commented 1 year ago

Some prow presets are based on credentials living in the build cluster running inside Google infrastructure. I would to sync them to k8s-infra-prow-build GCP project so we can move some jobs (e.g. kOps) to the community-owned infrastructure.

Some presets that need sync:

We could also add new credentials on k8s-infra but I have no idea who have access to those AWS accounts.

/sig k8s-infra /area infra /milestone v1.26

/assign @chaodaiG cc @cjwagner

k8s-ci-robot commented 1 year ago

@ameukam: The label(s) area/infra cannot be applied, because the repository doesn't have them.

In response to [this](https://github.com/kubernetes/test-infra/issues/27861): >Some prow presets are based on credentials living in the build cluster running inside Google infrastructure. I would to sync them to `k8s-infra-prow-build` GCP project so we can move some jobs (e.g. kOps) to the community-owned infrastructure. > >Some presets that need sync: > >- preset-aws-credential >- preset-aws-credential-aws-oss-testing >- preset-aws-ssh > >We could also add new credentials on k8s-infra but I have no idea who have access to those AWS accounts. > >/sig k8s-infra >/area infra >/milestone v1.26 > >/assign @chaodaiG >cc @cjwagner > > Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
k8s-ci-robot commented 1 year ago

@ameukam: The provided milestone is not valid for this repository. Milestones in this repository: [someday, v1.24, v1.25]

Use /milestone clear to clear the milestone.

In response to [this](https://github.com/kubernetes/test-infra/issues/27861): >Some prow presets are based on credentials living in the build cluster running inside Google infrastructure. I would to sync them to `k8s-infra-prow-build` GCP project so we can move some jobs (e.g. kOps) to the community-owned infrastructure. > >Some presets that need sync: > >- preset-aws-credential >- preset-aws-credential-aws-oss-testing >- preset-aws-ssh > >We could also add new credentials on k8s-infra but I have no idea who have access to those AWS accounts. > >/sig k8s-infra >/area infra >/milestone v1.26 > >/assign @chaodaiG >cc @cjwagner > > Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
chaodaiG commented 1 year ago

k8s-prow-builds cluster is the default build cluster for k8s prow:

$ k --context=k8s-prow-builds -ntest-pods get secrets | grep Opaque
akeyless-test-cred                                 Opaque                                1      212d
aws-cred                                           Opaque                                1      5y58d
aws-cred-new                                       Opaque                                1      4y358d
aws-credentials-607362164682                       Opaque                                1      3y283d
aws-credentials-768319786644                       Opaque                                1      3y283d
aws-ssh-key-secret                                 Opaque                                2      5y42d
azure-cred                                         Opaque                                1      714d
azure-secrets-store-cred                           Opaque                                4      2y213d
azure-ssh                                          Opaque                                1      3y352d
cadvisor-docker-credential                         Opaque                                2      4y266d
cadvisor-service-account                           Opaque                                1      4y289d
capv-ci-overrides                                  Opaque                                4      280d
capv-ipam-kubeconfig                               Opaque                                1      447d
cloud-provider-azure-account                       Opaque                                2      4y169d
cloud-provider-vsphere-e2e-config                  Opaque                                2      3y234d
cluster-api-provider-digitalocean-token            Opaque                                1      2y177d
cluster-api-provider-vsphere-gcs-prow              Opaque                                1      3y102d
cluster-api-provider-vsphere-vpn-config            Opaque                                5      2y300d
cluster-lifecycle-github-token                     Opaque                                1      2y38d
clusterapi-provider-vsphere-ci-prow                Opaque                                5      3y338d
eks-aws-credentials                                Opaque                                1      4y7d
fejta-bot-token                                    Opaque                                1      5y98d
gke-alpha-service-account                          Opaque                                1      5y72d
http-cookiefile                                    Opaque                                1      4y49d
ingress-nginx-codecov-token                        Opaque                                1      3y141d
istio-service-account                              Opaque                                1      4y204d
k8s-aws-alb-ingress-coveralls-token                Opaque                                1      4y1d
k8s-cip-test-prod-service-account                  Opaque                                1      3y74d
k8s-gcr-audit-test-prod-service-account            Opaque                                1      2y284d
k8s-minikube-build-gcs                             Opaque                                1      4y4d
k8s-multicluster-ingress-coveralls-token           Opaque                                1      4y353d
kops-e2e-do-ssh-key                                Opaque                                2      608d
node-feature-discovery-ci                          Opaque                                2      2y54d
service-account                                    Opaque                                1      3y165d
sig-storage-local-static-provisioner-pusher        Opaque                                1      3y289d
slack-tempelis-auth                                Opaque                                1      3y204d
spaces-digitalocean-s3                             Opaque                                2      644d
ssh-key-secret                                     Opaque                                2      5y143d
triage-service-account                             Opaque                                1      5y143d
velodrome-influxdb                                 Opaque                                1      5y143d
windows-private-registry-docker-config             Opaque                                1      2y97d

@ameukam , do we know which SA are wanted?

chaodaiG commented 1 year ago

nvm, found it:

- labels:
    preset-aws-credential: "true"
      secretName: aws-credentials-768319786644

# Credentials for using AWS test account 607362164682. Used for kops/eks tests.
- labels:
    preset-aws-credential-aws-oss-testing: "true"
      secretName: aws-credentials-607362164682

- labels:
    preset-aws-ssh: "true"
      secretName: aws-ssh-key-secret
chaodaiG commented 1 year ago

I have backed up all secrets from k8s-prow-builds cluster into GCP secrets manager in the k8s-prow-builds GCP project by running https://github.com/kubernetes/test-infra/tree/master/experiment/clustersecretbackup:

go run ./experiment/clustersecretbackup --cluster-context=gke_k8s-prow-builds_us-central1-f_prow --project=k8s-prow-builds --namespace=test-pods

The corresponding secrets for these 3 cluster secrets were mirrored in the GCP project where k8s infra build cluster is located at k8s-infra-prow-build.

ameukam commented 1 year ago

Can confirm secrets are copied.

gcloud secrets list --project k8s-infra-prow-build --format='table(name)'
NAME: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-credentials-607362164682

NAME: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-credentials-768319786644

NAME: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-ssh-key-secret

NAME: prow-build-service-account

NAME: prow-build-ssh-key-secret-ssh-private

NAME: prow-build-ssh-key-secret-ssh-public
ameukam commented 1 year ago

FYI @spiffxp

chaodaiG commented 1 year ago

sounds like this is now fixed?

ameukam commented 1 year ago

sounds like this is now fixed?

Yes. I'll try to migrate a few jobs and see what's happening.

Thank you so much again! 🙏🏾

/close

k8s-ci-robot commented 1 year ago

@ameukam: Closing this issue.

In response to [this](https://github.com/kubernetes/test-infra/issues/27861#issuecomment-1299345345): >> sounds like this is now fixed? > >Yes. I'll try to migrate a few jobs and see what's happening. > >Thank you so much again! 🙏🏾 > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.