search

kubernetes / release

Release infrastructure for Kubernetes and related components
Apache License 2.0
485 stars 504 forks source link

Migrate away from the `kubernetes-release` bucket #3729

Open puerco opened 3 months ago

puerco commented 3 months ago

Context

Release artifact have historically been published to a GCS bucket called kubernetes-release, this bucket is not under community control. As of the week of Aug 19th the release artifacts are being served from a CDN backed by a community bucket.

Currently, the contents of kubernetes-release are mirrored to the community bucket for serving every two hours.

/cc @kubernetes/release-engineering

TODO

Identify processes that need to be moved

We need to comprehensively search our processes to find those relying on data from kubernetes-release. Once we have an idea of which ones are reading and/or writing to the google owned bucket, let's expand the lists below with those that need to be migrated.

Migrating to the community bucket involves two groups of tasks, let's expand these as we find them:

Kubernetes Release Process

Internal Processes and Tests

ameukam commented 3 months ago

We had many conversations about this in https://github.com/kubernetes/k8s.io/issues/2396.

BenTheElder commented 2 months ago

We should also confirm the GCB project being used.

We need to comprehensively search our processes to find those relying on data from kubernetes-release. Once we have an idea of which ones are reading and/or writing to the google owned bucket, let's expand the lists below with those that need to be migrated.

I believe expected writes are only krel? There's a constant in krel for the bucket.

For reads, we've already made a big push to point things at dl.k8s.io instead, if any more crop up we can fix them later as worst case they won't have new releases until they switch, and the new bucket is intentionally not public read (only through the CDN).

I think it should be:

ameukam commented 2 months ago

make sure krel GCB service account has write to the new bucket (should be done already but double check)

It's not done yet. krel leverage the GCB service agent of the kubernetes-release-test GCP project to cut releases.

BenTheElder commented 2 months ago

We should also migrate out of kubernetes-release-test which is in google.com to a project in kubernetes.io, but we could do that in two phases.

ameukam commented 2 months ago

We should also migrate out of kubernetes-release-test which is in google.com to a project in kubernetes.io, but we could do that in two phases.

See: https://github.com/kubernetes/release/issues/3425

ameukam commented 2 months ago

/kind feature /priority important-soon