Proposal: Creation of sandbox area

[justinsb]

This repo and the existing bash/terraform code has shown the value of community development and management of infrastructure, but there are some challenges that make contributing here harder than we would like it to be:

• It's difficult for contributors to stand up their own version of the infrastructure e.g. https://github.com/kubernetes/k8s.io/blob/4074c17f39ebea00a2e6705292cb7f40592d5291/infra/gcp/terraform/modules/gke-project/main.tf#L17-L18 This makes it harder for contributors to send quality PRs, which in turn takes its toll on reviewers.
• We don't really have mechanisms for previewing changes
• We don't have a way to safely roll out infrastructure changes across dev/stage/prod environments.
• It's difficult for people to say "yes" to a change, because merging changes here is higher risk even than merging to kubernetes/kubernetes (where there are multiple layers of automated testing before anything reaches a release) We have no e2e tests for our PRs.
• Terraform shows the value of declarative infrastructure, but is prone to the problems of excess parameterization and simply is less familiar to kubernetes users than kubernetes is.

Generally, I'd summarize by saying that we could better aspire to our community values, which are actually a really nice summary of how kubernetes was developed collaboratively.

I therefore propose we create a sandbox directory and environment(s) to aspire to solve these problems and enable "safe infrastructure PRs".

Obviously there is a lot of work to stand all this up. Possibly it could be done using the bash and terraform we have today (if someone wants to try, I'm very much in favor of them doing so). I'm personally interested in exploring ConfigConnector on GCP - and similar tools on other clouds like ACK on AWS - and could devote some time to this.

Concretely, I propose we create a sandbox/kubeinfra directory (or some such name) and start to figure out how to stand up the sort of processes I'm talking about here, where contributors can fork, make changes and try them locally, we can create e2e tests, run those e2e tests against infrastructure PRs, and then we can roll changes to production with confidence gained in "lower" environments. I'm happy to take the lead on some of these experiments, but I hope that by doing so in this repo we can develop this collaboratively and learn from each other.

[eddiezane]

/cc

[hh]

/cc

[dims]

+1 from me to experiment

[ameukam]

+1 from to experiment.

a good MVP would be management of the GCR staging projects. (e.g. k8s-staging-bom).

[ameukam]

Some conversations about this subject:

• https://github.com/kubernetes/k8s.io/issues/1734#issuecomment-859942528

• https://github.com/kubernetes/k8s.io/issues/1885:#3355

  • Demo : https://youtu.be/ksInwexR8rY?list=PL69nYSiGNLP2Ghq7VW8rFbMFoHwvORuDL&t=2817

  I can help with the effort if needed.

  /area infra /priority important-longterm /milestone v1.24

[ameukam]

/milestone v1.25

[ameukam]

/area infra /priority important-longterm

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[ameukam]

/remove-lifecycle rotten /milestone clear /lifecycle frozen