Open spiffxp opened 3 years ago
/area infra/auditing
As part of https://github.com/kubernetes/k8s.io/pull/1890 I'm looking at k8s-conform's services. This project should be storage-only, so definitely going to disable compute
k8s-conform:
intent:
- secretmanager.googleapis.com
- storage-api.googleapis.com
- storage-component.googleapis.com
enabled:
- compute.googleapis.com
- oslogin.googleapis.com
- secretmanager.googleapis.com
- storage-api.googleapis.com
- storage-component.googleapis.com
expected:
- secretmanager.googleapis.com
- storage-api.googleapis.com
- storage-component.googleapis.com
to_enable: []
to_disable:
- compute.googleapis.com
- oslogin.googleapis.com
https://github.com/kubernetes/k8s.io/pull/1890#issuecomment-816326889 removed extraneous services for k8s-conform
I was looking into whether terraform could make this any easier... and it does not. They used to have a google_project_services
resource, but ended up deprecating and deleting it, ref: https://github.com/hashicorp/terraform-provider-google/pull/4587
I'm iterating on an attempt to make ensure_only_services
work a little better over in https://github.com/kubernetes/k8s.io/pull/2016 to close out https://github.com/kubernetes/k8s.io/issues/1963
/kind cleanup /priority important-longterm /assign I'll self assign since I'm noodling. But I could use some help, especially suggestions or prior art.
Ultimately I think the future here is a golang tool or something akin to crossplane rather than the ludicrous bash/jq I have going on
Ultimately I think the future here is a golang tool or something akin to crossplane rather than the ludicrous bash/jq I have going on
I started to take a look at CRDs provided by config-connector : https://cloud.google.com/config-connector/docs/concepts/resources. Planned to do a parallel to #1885.
From a freshly provisioned project (ref: https://github.com/kubernetes/k8s.io/pull/2195#issuecomment-861823341)
The existing bash/jq is not walking deps enough. None of the services should be unexpected after this first call.
plan to enable/disable the following services
to_enable:
- compute.googleapis.com
- containerregistry.googleapis.com
- logging.googleapis.com
- monitoring.googleapis.com
- storage-component.googleapis.com
to_disable: []
Operation "operations/acf.p2-783965148085-e3561dac-7684-4ec5-8a4e-087848353038" finished successfully.
Operation "operations/acf.p2-783965148085-73b68eda-a5fc-45e5-a827-90df93bfa3ef" finished successfully.
Operation "operations/acf.p2-783965148085-69a4fc7c-d39b-44da-9978-eb05e6bd4abc" finished successfully.
Operation "operations/acf.p2-783965148085-17922383-9f5a-4e41-b070-fb6d703dc45f" finished successfully.
Operation "operations/acf.p2-783965148085-de4f0675-e9d1-4b66-8fd6-9ecf662d56cd" finished successfully.
"INFO: dry-run mode, would run:" gcloud services disable --force oslogin.googleapis.com --project=k8s-infra-e2e-scale-5k-project
"INFO: dry-run mode, would run:" gcloud services disable --force pubsub.googleapis.com --project=k8s-infra-e2e-scale-5k-project
"INFO: dry-run mode, would run:" gcloud services disable --force storage-api.googleapis.com --project=k8s-infra-e2e-scale-5k-project
WARN: ensure_only_services: after enable/disable cycle, still projects to enable/disable: k8s-infra-e2e-scale-5k-project
intent:
- compute.googleapis.com
- containerregistry.googleapis.com
- logging.googleapis.com
- monitoring.googleapis.com
- storage-component.googleapis.com
enabled:
- compute.googleapis.com
- containerregistry.googleapis.com
- logging.googleapis.com
- monitoring.googleapis.com
- oslogin.googleapis.com
- pubsub.googleapis.com
- storage-api.googleapis.com
- storage-component.googleapis.com
expected:
- compute.googleapis.com
- containerregistry.googleapis.com
- logging.googleapis.com
- monitoring.googleapis.com
- storage-component.googleapis.com
to_enable: []
to_disable:
- oslogin.googleapis.com
- pubsub.googleapis.com
- storage-api.googleapis.com
I have a much gnarler set of jq lurking somewhere, I've been afraid to PR it since maybe it takes us too far in an unmaintainable direction. But I'm pretty sure it handles this correctly. I'll see if I can dig it up.
So I don't know when this happened, but it appears that gcloud services list
no longer includes dependency information.
That's gonna put a real damper on automating an audit of this.
/milestone v1.23 I would like to at least get a picture of how random the enabled services are
/milestone v1.24
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale /lifecycle frozen /milestone clear
This is a followup to https://github.com/kubernetes/k8s.io/issues/1675 which was focused just on k8s-staging-* GCP projects
While I haven't found docs for it, I am told that over-privileged users (e.g. organization admins) who use the UI are capable of auto-enabling services as they click from project-to-project or service-to-service. I think I often run into this when I'm using a service in my personal project, and then switch to a kubernetes.io project.
https://github.com/kubernetes/k8s.io/pull/1859 adds an
ensure_only_services "${project}" "${services[@]}"
function to auto-disable services that were directly specified, or direct dependencies of what was specified. I wasn't comfortable enabling it by default for everything, and I didn't want to rewrite ensure-prod-storage.sh just yet.This issue is intended to cover: