search

GoogleCloudPlatform / pubsec-declarative-toolkit

The GCP PubSec Declarative Toolkit is a collection of declarative solutions to help you on your Journey to Google Cloud. Solutions are designed using Config Connector and deployed using Config Controller.
Apache License 2.0
30 stars 27 forks source link

Developer Docs: expand on triaging/fixing failed cnrm-controller services deployed via kpt live apply #800

Open obriensystems opened 5 months ago

obriensystems commented 5 months ago

Draft docs in https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/wiki/DevOps#continue-kpt-fn-render-after-failed-services-fixed

need to go into https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/main/docs/landing-zone-v2/README.md

reference https://cloud.google.com/config-connector/docs/how-to/monitoring-your-resources#listing_all_resources https://cloud.google.com/architecture/managing-cloud-infrastructure-using-kpt#applying_a_kpt_package

triage:

michael@cloudshell:~/kcc-cso/kpt (kcc-cso-4380)$ kpt pkg tree core-landing-zone
Package "core-landing-zone"
├── [Kptfile]  Kptfile core-landing-zone
├── [resourcegroup.yaml]  ResourceGroup config-control/inventory-49821483
├── [setters.yaml]  ConfigMap setters
├── audits
│   ├── [folder.yaml]  Folder hierarchy/audits
│   └── logging-project
│       ├── [cloud-logging-buckets.yaml]  LoggingLogBucket logging/platform-and-component-log-bucket-cso1
│       ├── [cloud-logging-buckets.yaml]  LoggingLogBucket logging/security-log-bucket
│       ├── [cloud-storage-buckets.yaml]  StorageBucket logging/security-incident-log-bucket
│       ├── [project-iam.yaml]  IAMAuditConfig projects/logging-project-data-access-log-config
│       ├── [project-iam.yaml]  IAMPartialPolicy projects/mgmt-project-cluster-platform-and-component-log-bucket-writer-permissions
│       ├── [project-iam.yaml]  IAMPartialPolicy projects/platform-and-component-services-infra-log-bucket-writer-permissions
│       ├── [project-iam.yaml]  IAMPartialPolicy projects/platform-and-component-services-log-bucket-writer-permissions
│       ├── [project-iam.yaml]  IAMPartialPolicy projects/security-log-bucket-writer-permissions
│       ├── [project-sink.yaml]  LoggingLogSink logging/logging-project-cso1-data-access-sink
│       ├── [project.yaml]  Project projects/logging-project-cso1
│       ├── [services.yaml]  Service projects/logging-project-cso1-logging
│       ├── [services.yaml]  Service projects/logging-project-cso1-monitoring
│       └── monitoring
│           └── [metrics-scope.yaml]  MonitoringMonitoredProject logging/kcc-cso-4380
├── clients
│   └── [folder.yaml]  Folder hierarchy/clients
├── services
│   ├── [folder-sink.yaml]  LoggingLogSink logging/platform-and-component-services-log-sink
│   ├── [folder.yaml]  Folder hierarchy/services
│   └── services-infrastructure
│       ├── [folder-sink.yaml]  LoggingLogSink logging/platform-and-component-services-infra-log-sink
│       ├── [folder.yaml]  Folder hierarchy/services-infrastructure
│       └── dns-project
│           ├── [dns.yaml]  DNSManagedZone networking/dns-project-cso1-standard-core-public-dns
│           ├── [project.yaml]  Project projects/dns-project-cso1
│           └── [services.yaml]  Service projects/dns-project-cso1-dns
├── mgmt-project
│   ├── [project-sink.yaml]  LoggingLogSink logging/mgmt-project-cluster-platform-and-component-log-sink
│   ├── [services.yaml]  Service config-control/kcc-cso-4380-accesscontextmanager
│   ├── [services.yaml]  Service config-control/kcc-cso-4380-anthos
│   ├── [services.yaml]  Service config-control/kcc-cso-4380-cloudbilling
│   ├── [services.yaml]  Service config-control/kcc-cso-4380-cloudresourcemanager
│   ├── [services.yaml]  Service config-control/kcc-cso-4380-serviceusage
│   └── org-policies
│       ├── [compute-disable-serial-port-logging-except-mgt-project.yaml]  ResourceManagerPolicy policies/compute-disable-serial-port-logging-except-kcc-cso-4380
│       ├── [compute-require-shielded-vm-except-mgmt-project.yaml]  ResourceManagerPolicy policies/compute-require-shielded-vm-except-kcc-cso-4380
│       └── [compute-restrict-cloud-nat-usage-except-mgt-project.yaml]  ResourceManagerPolicy policies/compute-restrict-cloud-nat-usage-except-kcc-cso-4380
├── namespaces
│   ├── [config-management-monitoring.yaml]  IAMServiceAccount config-control/config-mgmt-mon-default-sa
│   ├── [config-management-monitoring.yaml]  IAMPolicyMember config-control/config-mgmt-mon-default-sa-metric-writer-permissions
│   ├── [config-management-monitoring.yaml]  IAMPartialPolicy config-control/config-mgmt-mon-default-sa-workload-identity-binding
│   ├── [config-management-monitoring.yaml]  ConfigConnectorContext config-management-monitoring/configconnectorcontext.core.cnrm.cloud.google.com
│   ├── [gatekeeper-system.yaml]  IAMServiceAccount config-control/gatekeeper-admin-sa
│   ├── [gatekeeper-system.yaml]  IAMPolicyMember config-control/gatekeeper-admin-sa-metric-writer-permissions
│   ├── [gatekeeper-system.yaml]  IAMPartialPolicy config-control/gatekeeper-admin-sa-workload-identity-binding
│   ├── [gatekeeper-system.yaml]  ConfigConnectorContext gatekeeper-system/configconnectorcontext.core.cnrm.cloud.google.com
│   ├── [hierarchy.yaml]  Namespace hierarchy
│   ├── [hierarchy.yaml]  IAMServiceAccount config-control/hierarchy-sa
│   ├── [hierarchy.yaml]  IAMPolicyMember config-control/hierarchy-sa-folderadmin-permissions
│   ├── [hierarchy.yaml]  IAMPartialPolicy config-control/hierarchy-sa-workload-identity-binding
│   ├── [hierarchy.yaml]  RoleBinding hierarchy/allow-folders-resource-reference-to-logging
│   ├── [hierarchy.yaml]  RoleBinding hierarchy/allow-hierarchy-resource-reference-from-config-control
│   ├── [hierarchy.yaml]  RoleBinding hierarchy/allow-hierarchy-resource-reference-from-policies
│   ├── [hierarchy.yaml]  RoleBinding hierarchy/allow-hierarchy-resource-reference-from-projects
│   ├── [hierarchy.yaml]  ConfigConnectorContext hierarchy/configconnectorcontext.core.cnrm.cloud.google.com
│   ├── [logging.yaml]  Namespace logging
│   ├── [logging.yaml]  IAMServiceAccount config-control/logging-sa
│   ├── [logging.yaml]  IAMPolicyMember config-control/logging-sa-logadmin-permissions
│   ├── [logging.yaml]  IAMPolicyMember config-control/logging-sa-monitoring-admin-kcc-cso-4380-permissions
│   ├── [logging.yaml]  IAMPartialPolicy config-control/logging-sa-workload-identity-binding
│   ├── [logging.yaml]  RoleBinding logging/allow-logging-resource-reference-from-projects
│   ├── [logging.yaml]  ConfigConnectorContext logging/configconnectorcontext.core.cnrm.cloud.google.com
│   ├── [logging.yaml]  IAMPolicyMember projects/logging-sa-monitoring-admin-logging-project-cso1-permissions
│   ├── [logging.yaml]  IAMPolicyMember projects/logging-sa-storageadmin-logging-project-cso1-permissions
│   ├── [management-namespace.yaml]  IAMPolicyMember config-control/config-control-sa-management-project-editor-permissions
│   ├── [management-namespace.yaml]  IAMPolicyMember config-control/config-control-sa-management-project-serviceaccountadmin-permissions
│   ├── [management-namespace.yaml]  IAMPolicyMember config-control/config-control-sa-orgroleadmin-permissions
│   ├── [networking.yaml]  Namespace networking
│   ├── [networking.yaml]  IAMServiceAccount config-control/networking-sa
│   ├── [networking.yaml]  IAMPolicyMember config-control/networking-sa-dns-permissions
│   ├── [networking.yaml]  IAMPolicyMember config-control/networking-sa-networkadmin-permissions
│   ├── [networking.yaml]  IAMPolicyMember config-control/networking-sa-security-permissions
│   ├── [networking.yaml]  IAMPolicyMember config-control/networking-sa-service-control-org-permissions
│   ├── [networking.yaml]  IAMPolicyMember config-control/networking-sa-servicedirectoryeditor-permissions
│   ├── [networking.yaml]  IAMPartialPolicy config-control/networking-sa-workload-identity-binding
│   ├── [networking.yaml]  IAMPolicyMember config-control/networking-sa-xpnadmin-permissions
│   ├── [networking.yaml]  ConfigConnectorContext networking/configconnectorcontext.core.cnrm.cloud.google.com
│   ├── [policies.yaml]  Namespace policies
│   ├── [policies.yaml]  IAMServiceAccount config-control/policies-sa
│   ├── [policies.yaml]  IAMPolicyMember config-control/policies-sa-orgpolicyadmin-permissions
│   ├── [policies.yaml]  IAMPartialPolicy config-control/policies-sa-workload-identity-binding
│   ├── [policies.yaml]  ConfigConnectorContext policies/configconnectorcontext.core.cnrm.cloud.google.com
│   ├── [projects.yaml]  Namespace projects
│   ├── [projects.yaml]  IAMServiceAccount config-control/projects-sa
│   ├── [projects.yaml]  IAMPolicyMember config-control/projects-sa-billinguser-permissions
│   ├── [projects.yaml]  IAMPolicyMember config-control/projects-sa-projectcreator-permissions
│   ├── [projects.yaml]  IAMPolicyMember config-control/projects-sa-projectdeleter-permissions
│   ├── [projects.yaml]  IAMPolicyMember config-control/projects-sa-projectiamadmin-permissions
│   ├── [projects.yaml]  IAMPolicyMember config-control/projects-sa-projectmover-permissions
│   ├── [projects.yaml]  IAMPolicyMember config-control/projects-sa-serviceusageadmin-permissions
│   ├── [projects.yaml]  IAMPartialPolicy config-control/projects-sa-workload-identity-binding
│   ├── [projects.yaml]  RoleBinding projects/allow-projects-resource-reference-from-logging
│   ├── [projects.yaml]  RoleBinding projects/allow-projects-resource-reference-from-networking
│   ├── [projects.yaml]  RoleBinding projects/allow-projects-resource-reference-from-policies
│   └── [projects.yaml]  ConfigConnectorContext projects/configconnectorcontext.core.cnrm.cloud.google.com
└── org
    ├── [org-sink.yaml]  LoggingLogSink logging/org-log-sink-data-access-logging-project-cso1
    ├── [org-sink.yaml]  LoggingLogSink logging/org-log-sink-security-logging-project-cso1
    └── custom-roles
        ├── [gke-firewall-admin.yaml]  IAMCustomRole config-control/gke-firewall-admin
        ├── [tier2-dnsrecord-admin.yaml]  IAMCustomRole config-control/tier2-dnsrecord-admin
        ├── [tier2-vpcpeering-admin.yaml]  IAMCustomRole config-control/tier2-vpcpeering-admin
        ├── [tier3-dnsrecord-admin.yaml]  IAMCustomRole config-control/tier3-dnsrecord-admin
        ├── [tier3-firewallrule-admin.yaml]  IAMCustomRole config-control/tier3-firewallrule-admin
        ├── [tier3-subnetwork-admin.yaml]  IAMCustomRole config-control/tier3-subnetwork-admin
        ├── [tier3-vpcsc-admin.yaml]  IAMCustomRole config-control/tier3-vpcsc-admin
        └── [tier4-secretmanager-admin.yaml]  IAMCustomRole config-control/tier4-secretmanager-admin
obriensystems commented 5 months ago

convert between kpt and kubectl

michael@cloudshell:~/kcc-cso/kpt (kcc-cso-4380)$ kubectl get gcp -n projects iampartialpolicy.iam.cnrm.cloud.google.com/security-log-bucket-writer-permissions 85m False DependencyNotFound 85m

michael@cloudshell:~/kcc-cso/kpt (kcc-cso-4380)$ kubectl describe iampartialpolicy.iam.cnrm.cloud.google.com/security-log-bucket-writer-permissions -n projects

iampartialpolicy.iam.cnrm.cloud.google.com/projects/security-log-bucket-writer-permissions to iampartialpolicy.iam.cnrm.cloud.google.com/security-log-bucket-writer-permissions -n projects



reference
https://cloud.google.com/architecture/managing-cloud-infrastructure-using-kpt#applying_a_kpt_package
https://cloud.google.com/config-connector/docs/how-to/monitoring-your-resources#listing_all_resources