Migrate tooling off of gs://kubernetes-jenkins

kubernetes / test-infra

Test infrastructure for the Kubernetes project.

Apache License 2.0

3.83k stars 2.65k forks source link

Migrate tooling off of gs://kubernetes-jenkins #33381

Open michelle192837 opened 2 months ago

michelle192837 commented 2 months ago

Follow-up to the control plane migration (https://github.com/kubernetes/test-infra/issues/33350).

Tools like Kettle, TestGrid, and others read from gs://kubernetes-jenkins, but should switch to using the new bucket (gs://kubernetes-ci-logs) or something else if needed.

[x] Ensure results from kubernetes-jenkins are synced to kubernetes-ci-logs (believe this is already done as part of the control plane migration?)
[x] Migrate Kettle
[x] Migrate TestGrid
[x] Migrate other miscellaneous use
[x] K8s Infra Prow writes to kubernetes-ci-logs instead of kubernetes-jenkins
[ ] Remove access permissions from kubernetes-jenkins
[ ] Update references to kubernetes-jenkins across repos
- [ ] kubernetes-sigs / prow
- [ ] (any others?)

/assign /sig testing

jbpratt commented 2 months ago

/assign

michelle192837 commented 1 month ago

Other updates (copying from Slack):

Purge messages on k8s-infra-kettle subscription
Stop the kettle deployment in kubernetes-public temporarily
Update k8s.io deployment.yaml
- Auto-deploy should handle it once submitted?
- Update deployment.yaml in test-infra as well, but nothing should care about that file

Kettle is updated, looks like it is now pulling from the new subscription successfully according to logs. (I'm not sure if it's correctly acking finished.json files since I don't know if we have an example of that yet :thinking: So still confirming that)

Okay good, it's acknowledging finished.json. I think we're good to go on the Prow change then? I'm going to let the bucket sync run one more time then stop it temporarily, then we can submit the prow change and hopefully it all works smoothly? Run has complete and the sync job is disabled!

michelle192837 commented 3 weeks ago

Kettle has had recent issues (see Slack threads in testing-ops and sig-k8s-infra)

Ref: https://github.com/kubernetes/test-infra/pull/33628

https://github.com/kubernetes/k8s.io/pull/7454 should hopefully fix this. (And I'll make the change in test-infra to match just in case, though it shouldn't change anything).

BenTheElder commented 3 weeks ago

https://cs.k8s.io/?q=kubernetes-jenkins&i=nope&files=&excludeFiles=&repos=

there's a lot more of these than I hoped, but most of them are just links in docs. a few are pulling data. a few more don't need changing and are discussing the migration.