search

kubernetes / test-infra

Test infrastructure for the Kubernetes project.
Apache License 2.0
3.83k stars 2.65k forks source link

Kops - Migrate remaining upgrade jobs to k8s infra prow clusters #32885

Closed rifelpet closed 3 months ago

rifelpet commented 3 months ago

This migrates the remaining kops upgrade jobs to the k8s infra prow clusters. These jobs will fail because they use the k8s 1.24 / 1.25 / 1.26 e2e.test binaries which are built with an aws-sdk-go version too old to authenticate on the EKS prow clusters. Once we see the failures we can assess whether we can ignore specific test cases or whether the whole job wont work in which case we'll just delete these jobs.

Refs: https://github.com/kubernetes/kops/issues/16637 https://github.com/kubernetes/k8s.io/issues/5127 https://github.com/kubernetes/k8s.io/issues/2625

/cc @upodroid @hakman

k8s-ci-robot commented 3 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman, rifelpet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[config/jobs/kubernetes/kops/OWNERS](https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/kops/OWNERS)~~ [hakman,rifelpet] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
k8s-ci-robot commented 3 months ago

@rifelpet: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

In response to [this](https://github.com/kubernetes/test-infra/pull/32885): >This migrates the remaining kops upgrade jobs to the k8s infra prow clusters. These jobs will fail because they use the k8s 1.24 / 1.25 / 1.26 `e2e.test` binaries which are built with an aws-sdk-go version too old to authenticate on the EKS prow clusters. Once we see the failures we can assess whether we can ignore specific test cases or whether the whole job wont work in which case we'll just delete these jobs. > >Refs: https://github.com/kubernetes/kops/issues/16637 https://github.com/kubernetes/k8s.io/issues/5127 > >/cc @upodroid @hakman Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
upodroid commented 3 months ago

btw, we need to eliminate the dependency on the kops-ci bucket as that is no longer being updated after 1st of Aug.

k8s-staging-kops is the replacement bucket and we do have some jobs fetching kops from there

upodroid commented 3 months ago

I'll open a PR updating the marker locations

rifelpet commented 3 months ago

@upodroid which job publishes latest-ci.txt version marker to k8s-staging-kops? I see the marker files exist but I can't find the job that publishes them. I'd like to use it as a reference for migrating the release branch version markers to k8s-staging-kops.

upodroid commented 3 months ago

This job https://testgrid.k8s.io/sig-cluster-lifecycle-kops#kops-postsubmit-push-to-staging

rifelpet commented 3 months ago

Ah I see. This job doesn't create a kops cluster so the only access it needs is to the k8s-staging-kops bucket.

Any thoughts on how a job can both create AWS kops clusters and have write access to GCS? This is what blocks the presubmits and updown and pipeline jobs from being migrated to community-owned infra.

If that won't be possible, then we'd need the kops clusters to run in the same cloud provider as the artifact bucket, either move the jobs to create GCP clusters or create an S3 artifacts bucket. Moving the artifacts to S3 shouldn't be a problem because any consumers of the bucket will access it publicly and anonymously.