Open spiffxp opened 3 years ago
I'm getting close to having our existing job run hourly. I'm not sure where we would want them to run as post-submits. (any change to k8s.io?) I assume your wanting that to be able to type the changes via PRs into audit updates.
Glanced at Cloud Asset Inventory to look into something else
Two things that disqualify ~it~ gcloud asset search-all-resources
as a general-purpose solution for us, I think:
But, gcloud asset search-all-iam-policies
dumps IAM policy bindings really quickly, and for most of the resources we commonly we use:
# a prow cluster
$ gcloud asset search-all-iam-policies --scope=projects/k8s-infra-prow-build-trusted --format="value(resource)" | cut -d/ -f4-
projects/k8s-infra-prow-build-trusted/serviceAccounts/prow-deployer@k8s-infra-prow-build-trusted.iam.gserviceaccount.com
projects/k8s-infra-prow-build-trusted/serviceAccounts/gcb-builder@k8s-infra-prow-build-trusted.iam.gserviceaccount.com
projects/k8s-infra-prow-build-trusted/datasets/usage_metering_prow_build_trusted
projects/k8s-infra-prow-build-trusted/serviceAccounts/prow-build-trusted@k8s-infra-prow-build-trusted.iam.gserviceaccount.com
projects/k8s-infra-prow-build-trusted
# an e2e project
$ gcloud asset search-all-iam-policies --scope=projects/k8s-infra-e2e-boskos-001 --format="value(resource)" | cut -d/ -f4-
kubernetes-staging-485128143e-asia
kubernetes-staging-485128143e-eu
kubernetes-staging-485128143e
projects/k8s-infra-e2e-boskos-00
# a staging project
$ gcloud asset search-all-iam-policies --scope=projects/k8s-staging-e2e-test-images --format="value(resource)" | cut -d/ -f4-
k8s-staging-e2e-test-images-gcb
k8s-staging-e2e-test-images
artifacts.k8s-staging-e2e-test-images.appspot.com
projects/k8s-staging-e2e-test-images
So if nothing else, and excluding secrets, I could see this being useful to quickly audit/reconcile IAM polices across the org.
A next step would be to look at what sort of info is availabe from gcloud asset export
https://github.com/kubernetes/k8s.io/issues/1981 covers exploring cloud alpha resource-config bulk-export
https://github.com/kubernetes/test-infra/pull/22239 should update the audit job to only bump open PRs if there are new changes in the audit directory, which will hopefully cut down on open PRs with long trails of force-pushes that don't actually change the files that have been reviewed.
It currently takes about ballpark 80 minutes to perform a full audit: https://testgrid.k8s.io/wg-k8s-infra-k8sio#ci-k8sio-audit&width=20&graph-metrics=test-duration-minutes
I think we can do better.
gcloud asset list
is a thing now, if we want to try munging the yaml / json that dumps into the same format we're currently using, or make our yaml / json match its format
/milestone v1.23 /priority backlog That said I think speeding this up may be less important than moving things over.
/remove-priority important-longterm
/milestone v1.24
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale /lifecycle frozen /milestone clear
/milestone v1.32
EDIT: Opting to treat this as an umbrella issue instead of placeholder to noodle on ideas
An umbrella issue to capture ideas and suggestions to improve our audit process.
Currently:
link_to_issue_comment
"@foo
did you change something manually here?"Some problems with this:
TODO: flesh these out into issues? or just track a list here
Our audit results are not easily reconciled:
We can't audit or dump everything due to IAM issues:
roles/viewer
at the org level?Auditing dumps are too slow:
gcloud asset
gcloud resource-config bulk-export
- https://github.com/kubernetes/k8s.io/issues/1981Bugs with our audit script right now:
/wg k8s-infra /area infra/auditing /area access /priority important-longterm /kind cleanup
cc @dims @thockin @cblecker @hh