search

GoogleCloudPlatform / pubsec-declarative-toolkit

The GCP PubSec Declarative Toolkit is a collection of declarative solutions to help you on your Journey to Google Cloud. Solutions are designed using Config Connector and deployed using Config Controller.
Apache License 2.0
31 stars 28 forks source link

Add dev deployment.sh for KCC cluster CD create/delete and LZ initial deployment/delete with canary #192

Open fmichaelobrien opened 1 year ago

fmichaelobrien commented 1 year ago

sh changes in prep of adding delete to Arete shortly

pending

start with

export REGION=northamerica-northeast1
export SUBNET=pdt-na1-sn
export CLUSTER=pdt-na1
export NETWORK=pdt-na1-vpc
export CC_PROJECT_ID=controller-agz-1201
export BOOT_PROJECT_ID=$(gcloud config list --format 'value(core.project)')
export BILLING_ID=$(gcloud alpha billing projects describe $BOOT_PROJECT_ID '--format=value(billingAccountName)' | sed 's/.*\///')
USER="$(kubectl get ConfigConnectorContext -n config-control -o jsonpath='{.items[0].spec.googleServiceAccount}' 2> /dev/null)"
ORGID=$(gcloud projects get-ancestors $CC_PROJECT_ID --format='get(id)' | tail -1)
gcloud compute networks create $NETWORK --subnet-mode=custom
gcloud compute networks subnets create $SUBNET --network $NETWORK --range 192.168.0.0/16 --region $REGION
gcloud alpha anthos config controller create $CLUSTER --location $REGION --network $NETWORK --subnet $SUBNET --full-management
gcloud anthos config controller list
gcloud anthos config controller delete --location $REGION $CLUSTER --quiet
oot_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (controller-agz-1201)$ ./bootstrap.sh run
Start: 1669999560
BILLING_ID: 011BCB-037F97-C9169E
ORGID: 6839210352
Create request issued for: [pdt-na1a]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1669999564244-5eedb12cd46ba-b1d78310-a897d51f] to complete...working...
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1669999564244-5eedb12cd46ba-b1d78310-a897d51f] to complete...done.     
Created instance [pdt-na1a].
Fetching cluster endpoint and auth data.
kubeconfig entry generated for krmapihost-pdt-na1a.
Cluster create time: 1105
Create request issued for: [pdt-na1b]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670000669112-5eedb54a83894-65a57b7a-0a500016] to complete...done.     
Created instance [pdt-na1b].
Fetching cluster endpoint and auth data.
kubeconfig entry generated for krmapihost-pdt-na1b.
Cluster create time: 1105
Create request issued for: [pdt-na1c]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670001774021-5eedb9683c996-e79796ea-d4a84512] to complete...failed.   
ERROR: (gcloud.alpha.anthos.config.controller.create) unexpected error occurred while waiting for SLM operation [projects/krmapihosting-slm/locations/northamerica-northeast1/operations/operation-1670001779009-5eedb96cfe9b4-581aecfb-9cfebea7]: You cannot create more than 3 clusters in location northamerica-northeast1; to create more than 3, you must request an increase of your Google Compute Engine quota for region northamerica-northeast1 to 25 CPUs or more.
obriensystems commented 1 year ago

testing vpc/kcc creation

root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$ ./bootstrap.sh -b pubsec-declarative-agz -u pdt1 -c true -l true -d false -p controller-agz-1201
Date: Sun 04 Dec 2022 02:07:31 AM UTC
Timestamp: 1670119651
running with: -b pubsec-declarative-agz -u pdt1 -c true -l true -d false -p controller-agz-1201
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
Start: 1670119652
unique string: pdt1
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
SUBNET: pdt-pdt1-sn
CLUSTER: pdt-pdt1
CC_PROJECT_ID: controller-agz-1201
BOOT_PROJECT_ID: pubsec-declarative-agz
BILLING_ID: 011D7E-BD499C-CF71C5
ORGID: 6839210352
Updated property [core/project].
Switched to KCC project controller-agz-1201
Create VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/global/networks/pdt-pdt1-vpc].
NAME: pdt-pdt1-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp:22,tcp:3389,icmp
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp:22,tcp:3389,icmp

Create subnet pdt-pdt1-sn off VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/regions/northamerica-northeast1/subnetworks/pdt-pdt1-sn].
NAME: pdt-pdt1-sn
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
RANGE: 192.168.0.0/16
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Creating Anthos KCC autopilot cluster pdt-pdt1 in region northamerica-northeast1 in subnet pdt-pdt1-sn off VPC pdt-pdt1-vpc
Create request issued for: [pdt-pdt1]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670119679281-5eef70a374862-ac53f749-0ea158b8] to complete...working...
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670119679281-5eef70a374862-ac53f749-0ea158b8] to complete...done.ng...
Created instance [pdt-pdt1].
Fetching cluster endpoint and auth data.
kubeconfig entry generated for krmapihost-pdt-pdt1.
Cluster create time: 1105 sec
List Clusters:
NAME: pdt-pdt1
LOCATION: northamerica-northeast1
STATE: RUNNING
Total Duration: 1132 sec
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
**** Done ****
root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$

testing vpc/kcc creation+deletion run 1 - intermittent crash

Creating Anthos KCC autopilot cluster pdt-pdt1 in region northamerica-northeast1 in subnet pdt-pdt1-sn off VPC pdt-pdt1-vpc
Create request issued for: [pdt-pdt1]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670122093720-5eef79a20b2ed-8864d149-dd378e85] to complete...working.. 
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670122093720-5eef79a20b2ed-8864d149-dd378e85] to complete...failed.   
ERROR: gcloud crashed (TypeError): string indices must be integers

If you would like to report this issue, please run the following command:
  gcloud feedback

To check gcloud for common probl

run 2

root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (controller-agz-1201)$ ./bootstrap.sh -b pubsec-declarative-agz -u pdt1 -c false -l true -d true -p controller-agz-1201
Date: Sun 04 Dec 2022 03:00:43 AM UTC
Timestamp: 1670122843
running with: -b pubsec-declarative-agz -u pdt1 -c false -l true -d true -p controller-agz-1201
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
Start: 1670122844
unique string: pdt1
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
SUBNET: pdt-pdt1-sn
CLUSTER: pdt-pdt1
CC_PROJECT_ID: controller-agz-1201
BOOT_PROJECT_ID: pubsec-declarative-agz
BILLING_ID: 011D7E-BD499C-CF71C5
ORGID: 6839210352
Updated property [core/project].
Switched to KCC project controller-agz-1201
List Clusters:
NAME: pdt-pdt1
LOCATION: northamerica-northeast1
STATE: CREATING
Delete Cluster pdt-pdt1 in region northamerica-northeast1
Delete request issued for: [pdt-pdt1]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670122848935-5eef7c7245f12-09e6663f-649d3100] to complete...working...
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670122848935-5eef7c7245f12-09e6663f-649d3100] to complete...done.     
Deleted instance [pdt-pdt1].
Cluster delete time: 924 sec
deleting subnet pdt-pdt1-sn
Deleted [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/regions/northamerica-northeast1/subnetworks/pdt-pdt1-sn].
deleting vpc pdt-pdt1-vpc

Deleted [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/global/networks/pdt-pdt1-vpc].
Total Duration: 955 sec
Date: Sun 04 Dec 2022 03:16:39 AM UTC
Timestamp: 1670123799
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
**** Done ****
root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$

full run ok

root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$ ./bootstrap.sh -b pubsec-declarative-agz -u pdt1 -c true -l true -d true -p controller-agz-1201
Date: Sun 04 Dec 2022 03:17:47 AM UTC
Timestamp: 1670123867
running with: -b pubsec-declarative-agz -u pdt1 -c true -l true -d true -p controller-agz-1201
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
Start: 1670123868
unique string: pdt1
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
SUBNET: pdt-pdt1-sn
CLUSTER: pdt-pdt1
CC_PROJECT_ID: controller-agz-1201
BOOT_PROJECT_ID: pubsec-declarative-agz
BILLING_ID: 011D7E-BD499C-CF71C5
ORGID: 6839210352
Updated property [core/project].
Switched to KCC project controller-agz-1201
Create VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/global/networks/pdt-pdt1-vpc].
NAME: pdt-pdt1-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp:22,tcp:3389,icmp

Create subnet pdt-pdt1-sn off VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/regions/northamerica-northeast1/subnetworks/pdt-pdt1-sn].
NAME: pdt-pdt1-sn
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
RANGE: 192.168.0.0/16
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Creating Anthos KCC autopilot cluster pdt-pdt1 in region northamerica-northeast1 in subnet pdt-pdt1-sn off VPC pdt-pdt1-vpc
Create request issued for: [pdt-pdt1]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670123897653-5eef805a68732-a2dd9f18-70ce76be] to complete...working   
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670123897653-5eef805a68732-a2dd9f18-70ce76be] to complete...working.  
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670123897653-5eef805a68732-a2dd9f18-70ce76be] to complete...done.     
Created instance [pdt-pdt1].
Fetching cluster endpoint and auth data.
kubeconfig entry generated for krmapihost-pdt-pdt1.
Cluster create time: 1105 sec
List Clusters:
NAME: pdt-pdt1
LOCATION: northamerica-northeast1
STATE: RUNNING
Delete Cluster pdt-pdt1 in region northamerica-northeast1
Delete request issued for: [pdt-pdt1]
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670125003478-5eef847901705-90566a76-7a654d0a] to complete...working...
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670125003478-5eef847901705-90566a76-7a654d0a] to complete...working.. 
Waiting for operation [projects/controller-agz-1201/locations/northamerica-northeast1/operations/operation-1670125003478-5eef847901705-90566a76-7a654d0a] to complete...done.     
Deleted instance [pdt-pdt1].
Cluster delete time: 404 sec
deleting subnet pdt-pdt1-sn
Deleted [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/regions/northamerica-northeast1/subnetworks/pdt-pdt1-sn].
deleting vpc pdt-pdt1-vpc
Deleted [https://www.googleapis.com/compute/v1/projects/controller-agz-1201/global/networks/pdt-pdt1-vpc].
Total Duration: 1565 sec
Date: Sun 04 Dec 2022 03:43:53 AM UTC
Timestamp: 1670125433
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
**** Done ****
root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$
obriensystems commented 1 year ago

with project/billing create/delete

root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$ ./deployment.sh -b pubsec-declarative-agz -u pdt1 -c true -l true -d true -p controller-agz
Date: Mon 05 Dec 2022 03:51:22 AM UTC
Timestamp: 1670212282
running with: -b pubsec-declarative-agz -u pdt1 -c true -l true -d true -p controller-agz
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
Start: 1670212283
unique string: pdt1
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
SUBNET: pdt-pdt1-sn
CLUSTER: pdt-pdt1
CC_PROJECT_ID: controller-agz-1560
BOOT_PROJECT_ID: pubsec-declarative-agz
BILLING_ID: 011D7E-BD499C-CF71C5
ORGID: 6839210352
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/controller-agz-1560].
Waiting for [operations/cp.8148807460816492413] to finish...done.    
Enabling service [cloudapis.googleapis.com] on project [controller-agz-1560]...
Operation "operations/acat.p2-689469500658-76e49192-dc6f-4438-a9b4-ef9f284ae843" finished successfully.
Updated property [core/project] to [controller-agz-1560].
Created KCC project: controller-agz-1560
Updated property [core/project].
billingAccountName: billingAccounts/011D7E-BD499C-CF71C5
billingEnabled: true
name: projects/controller-agz-1560/billingInfo
projectId: controller-agz-1560
Enabling APIs
Operation "operations/acf.p2-689469500658-fd586f8c-5d0f-4336-8160-27a19e5689ad" finished successfully.
Create VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/controller-agz-1560/global/networks/pdt-pdt1-vpc].
NAME: pdt-pdt1-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp:22,tcp:3389,icmp

Create subnet pdt-pdt1-sn off VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/controller-agz-1560/regions/northamerica-northeast1/subnetworks/pdt-pdt1-sn].
NAME: pdt-pdt1-sn
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
RANGE: 192.168.0.0/16
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Creating Anthos KCC autopilot cluster pdt-pdt1 in region northamerica-northeast1 in subnet pdt-pdt1-sn off VPC pdt-pdt1-vpc
Create request issued for: [pdt-pdt1]
Waiting for operation [projects/controller-agz-1560/locations/northamerica-northeast1/operations/operation-1670212402324-5ef0ca0f0a353-77836d30-6d9685ea] to complete...working   
Waiting for operation [projects/controller-agz-1560/locations/northamerica-northeast1/operations/operation-1670212402324-5ef0ca0f0a353-77836d30-6d9685ea] to complete...working...
Waiting for operation [projects/controller-agz-1560/locations/northamerica-northeast1/operations/operation-1670212402324-5ef0ca0f0a353-77836d30-6d9685ea] to complete...done.     
Created instance [pdt-pdt1].
Fetching cluster endpoint and auth data.
kubeconfig entry generated for krmapihost-pdt-pdt1.
Cluster create time: 1105 sec
List Clusters:
NAME: pdt-pdt1
LOCATION: northamerica-northeast1
STATE: RUNNING
Delete Cluster pdt-pdt1 in region northamerica-northeast1
Delete request issued for: [pdt-pdt1]
Waiting for operation [projects/controller-agz-1560/locations/northamerica-northeast1/operations/operation-1670213508563-5ef0ce2e081c1-7fe4454c-f00bbaae] to complete...done.     
Deleted instance [pdt-pdt1].
Cluster delete time: 405 sec
deleting subnet pdt-pdt1-sn
Deleted [https://www.googleapis.com/compute/v1/projects/controller-agz-1560/regions/northamerica-northeast1/subnetworks/pdt-pdt1-sn].
deleting vpc pdt-pdt1-vpc
Deleted [https://www.googleapis.com/compute/v1/projects/controller-agz-1560/global/networks/pdt-pdt1-vpc].
billingAccountName: ''
billingEnabled: false
name: projects/controller-agz-1560/billingInfo
projectId: controller-agz-1560
Deleted [https://cloudresourcemanager.googleapis.com/v1/projects/controller-agz-1560].

You can undo this operation for a limited period by running the command below.
    $ gcloud projects undelete controller-agz-1560

See https://cloud.google.com/resource-manager/docs/creating-managing-projects for information on shutting down projects.
Total Duration: 1664 sec
Date: Mon 05 Dec 2022 04:19:07 AM UTC
Timestamp: 1670213947
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
obriensystems commented 1 year ago

added kpt pkg get 

Updated IAM policy for organization [583675367868].
Package "landing-zone":
Fetching https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit@main
From https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit
 * branch            main       -> FETCH_HEAD
 + 9750a11...af9d2b2 main       -> origin/main  (forced update)
Adding package "solutions/landing-zone".

Fetched 1 package(s).

add instructions for known constraints.yaml https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/issues/112

michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kpt live apply landing-zone --reconcile-timeout=2m --output=table
I1207 18:46:29.264183   11983 request.go:601] Waited for 1.198285577s due to client-side throttling, not priority and fairness, request: GET:https://35.203.38.53/apis/status.gatekeeper.sh/v1beta1?timeout=32s
Error: 4 resource types could not be found in the cluster or as CRDs among the applied resources.

Resource types:
constraints.gatekeeper.sh/v1beta1, Kind=NamingPolicy
constraints.gatekeeper.sh/v1beta1, Kind=DataLocation
constraints.gatekeeper.sh/v1beta1, Kind=LimitEgressTraffic
constraints.gatekeeper.sh/v1beta1, Kind=CloudMarketPlaceConfig

add workaround in root .krmignore +constraint.yaml

triage errors as follow

kubectl get gcp
NAME                                                                                               AGE   READY   STATUS         STATUS AGE
[resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-contact-domains](http://resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-contact-domains)                10m   False   UpdateFailed   10m

michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kubectl describe [resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-contact-domains](http://resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-contact-domains)

   Message:               Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Organization policy for organizations/0000000000: googleapi: Error 403: The caller does not have permission, f

My kpt render is not taking effect - like it did last time - missing something 

    external: "0000000000" # kpt-set: ${org-id}

forgot the middle kpt fn render between the init and apply - working

michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kpt live init landing-zone --namespace config-control
initializing "resourcegroup.yaml" data (namespace: config-control)...failed
Error: Inventory information has already been added to the package ResourceGroup object. Changing it after a package has been applied to the cluster can lead to undesired results. Use the --force flag to suppress this error.
michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kpt live init landing-zone --namespace config-control --force
initializing "resourcegroup.yaml" data (namespace: config-control)...success
michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kpt fn render landing-zone
Package "landing-zone/environments/common/guardrails-policies":
Package "landing-zone/environments/common":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 2.5s
  Results:
    [info]: namespace "common" updated to "config-control", 23 value(s) changed

Package "landing-zone/environments/nonprod":
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 300ms
  Results:
    [info]: namespace "nonprod" updated to "config-control", 7 value(s) changed

Package "landing-zone/environments/prod":
[RUNNING] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0"
[PASS] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0" in 3.2s
  Results:
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-compute: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-logging: generated service
[RUNNING] "gcr.io/kpt-fn/set-namespace:v0.4.1"
[PASS] "gcr.io/kpt-fn/set-namespace:v0.4.1" in 300ms
  Results:
    [info]: namespace "prod" updated to "config-control", 4 value(s) changed

Package "landing-zone":
[RUNNING] "gcr.io/kpt-fn/apply-setters:v0.2"
[PASS] "gcr.io/kpt-fn/apply-setters:v0.2" in 2.2s
  Results:
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "583675367868"
    [info] metadata.annotations.cnrm.cloud.google.com/organization-id: set field value to "583675367868"
    [info] spec.projectID: set field value to "net-per-prj-common-oldv1"
    [info] spec.parentRef.external: set field value to "583675367868"
    ...(87 line(s) truncated, use '--truncate-output=false' to disable)
[RUNNING] "gcr.io/kpt-fn/generate-folders:v0.1.1"
[PASS] "gcr.io/kpt-fn/generate-folders:v0.1.1" in 5.1s
[RUNNING] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0"
[PASS] "gcr.io/kpt-fn/enable-gcp-services:v0.1.0" in 1.8s
  Results:
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/nonprod-nethost-service-compute: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/nonprod-nethost-service-dns: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/nonprod-nethost-service-logging: generated service
    [info] serviceusage.cnrm.cloud.google.com/v1beta1/Service/config-control/prod-nethost-service-compute: recreated service
    ...(3 line(s) truncated, use '--truncate-output=false' to disable)
[RUNNING] "gcr.io/kpt-fn/gatekeeper:v0.2.1"
[PASS] "gcr.io/kpt-fn/gatekeeper:v0.2.1" in 3.7s
[RUNNING] "gcr.io/kpt-fn/kubeval:v0.3.0"
[PASS] "gcr.io/kpt-fn/kubeval:v0.3.0" in 22.5s

Successfully executed 9 function(s) in 5 package(s).
michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$

running live apply again

delete package
michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kpt live destroy landing-zone

recreate
michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kpt live init landing-zone --namespace config-control --force
already did the render before
michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$ kpt live apply landing-zone --reconcile-timeout=2m --output=tabl
NAMESPACE   RESOURCE                                  ACTION        STATUS      RECONCILED  CONDITIONS                                AGE     MESSAGE
            ConstraintTemplate/cloudmarketplaceconfi  Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/datalocation           Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/limitegresstraffic     Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/namingpolicy           Skipped       Current                 <None>                                    87m     Resource is current
config-con  ConfigMap/setters                         Skipped       Current                 <None>                                    88m     Resource is always ready
config-con  AccessContextManagerAccessLevel/commonac  Skipped       InProgress              Ready                                     88m     reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessLevel/nonprodp  Successful    InProgress              Ready                                     2m      reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessLevel/prodacce  Skipped       InProgress              Ready                                     88m     reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessPolicy/orgacce  Skipped       InProgress              Ready                                     88m     Update call failed: error applying desir
config-con  ComputeFirewall/allow-egress-internet     Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pr  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pu  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress         Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress-pr      Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingressp        Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/computefirewall-sample-d  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/prod-firewall-default-de  Skipped       InProgress              Ready                                     87m     reference ComputeNetwork config-control/
config-con  ComputeNetwork/common-ha-perimeter        Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/common-mgmt-perimeter      Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/nonprod-sharedvpc          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/priv-perimeter             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/prod-sharedvpc             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/public-perimeter           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeProjectMetadata/nonprod-oslogin-m  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeRoute/egress-internet-nonprod      Skipped       InProgress              Ready                                     87m     reference ComputeNetwork config-control/
config-con  ComputeRoute/egress-internet-prod         Skipped       InProgress              Ready                                     87m     reference ComputeNetwork config-control/
config-con  ComputeSharedVPCHostProject/computeshare  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ComputeSharedVPCHostProject/nonprod-shar  Skipped       InProgress              Ready                                     87m     Update call failed: error applying desir
config-con  ComputeSubnetwork/common-ha-perimeter-su  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/management              Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/nonprod-sharedvpc-subn  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/priv-perimeter-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/prod-sharedvpc-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/public-perimeter-subne  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  IAMPartialPolicy/audit-sink-writer        Skipped       InProgress              Ready                                     87m     reference StorageBucket config-control/a
config-con  IAMPartialPolicy/log-sink-writer          Skipped       InProgress              Ready                                     87m     reference StorageBucket config-control/l
config-con  IAMPolicyMember/audit-viewer              Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMPolicyMember/billing-iam-member        Successful    InProgress              Ready                                     2m      reference Project config-control/audit-p
config-con  IAMPolicyMember/log-reader                Skipped       InProgress              Ready                                     87m     Update call failed: error setting policy
config-con  IAMPolicyMember/log-writer                Skipped       InProgress              Ready                                     87m     Update call failed: error setting policy
config-con  IAMPolicyMember/organization-viewer       Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMServiceAccount/billing-service-accoun  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  LoggingLogSink/audit-bucket-sink          Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/a
config-con  LoggingLogSink/logs-bucket-sink           Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/l
config-con  Folder/audit-and-security                 Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.audit           Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.security        Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/automation                         Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking.nonprod  Successful    InProgress              Ready                                     2m      Update in progress
config-con  Folder/infrastructure.networking.prodnet  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.sharedinfrastructu  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/sandbox                            Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/shared-services                    Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads                          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.dev                      Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.prod                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.uat                      Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Project/audit-prj-id-oldv1                Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/guardrails-project-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-host-prj-nonprod-oldv1        Successful    InProgress              Ready                                     2m      reference Folder config-control/infrastr
config-con  Project/net-host-prj-prod-oldv1           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-per-prj-common-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ResourceManagerPolicy/allowed-contact-do  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/allowed-policy-mem  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-guest-attr  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-nested-vir  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-serial-por  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-serviceacc  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-vpc-extern  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/require-shielded-v  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/require-trusted-im  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
NAMESPACE   RESOURCE                                  ACTION        STATUS      RECONCILED  CONDITIONS                                AGE     MESSAGE
            ConstraintTemplate/cloudmarketplaceconfi  Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/datalocation           Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/limitegresstraffic     Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/namingpolicy           Skipped       Current                 <None>                                    87m     Resource is current
config-con  ConfigMap/setters                         Skipped       Current                 <None>                                    88m     Resource is always ready
config-con  AccessContextManagerAccessLevel/commonac  Skipped       InProgress              Ready                                     88m     reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessLevel/nonprodp  Successful    InProgress              Ready                                     2m      reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessLevel/prodacce  Skipped       InProgress              Ready                                     88m     reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessPolicy/orgacce  Skipped       InProgress              Ready                                     88m     Update call failed: error applying desir
config-con  ComputeFirewall/allow-egress-internet     Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pr  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pu  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress         Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress-pr      Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingressp        Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/computefirewall-sample-d  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/prod-firewall-default-de  Skipped       InProgress              Ready                                     88m     reference ComputeNetwork config-control/
config-con  ComputeNetwork/common-ha-perimeter        Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/common-mgmt-perimeter      Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/nonprod-sharedvpc          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/priv-perimeter             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/prod-sharedvpc             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/public-perimeter           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeProjectMetadata/nonprod-oslogin-m  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeRoute/egress-internet-nonprod      Skipped       InProgress              Ready                                     87m     reference ComputeNetwork config-control/
config-con  ComputeRoute/egress-internet-prod         Skipped       InProgress              Ready                                     87m     reference ComputeNetwork config-control/
config-con  ComputeSharedVPCHostProject/computeshare  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ComputeSharedVPCHostProject/nonprod-shar  Skipped       InProgress              Ready                                     87m     Update call failed: error applying desir
config-con  ComputeSubnetwork/common-ha-perimeter-su  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/management              Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/nonprod-sharedvpc-subn  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/priv-perimeter-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/prod-sharedvpc-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/public-perimeter-subne  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  IAMPartialPolicy/audit-sink-writer        Skipped       InProgress              Ready                                     87m     reference StorageBucket config-control/a
config-con  IAMPartialPolicy/log-sink-writer          Skipped       InProgress              Ready                                     87m     reference StorageBucket config-control/l
config-con  IAMPolicyMember/audit-viewer              Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMPolicyMember/billing-iam-member        Successful    InProgress              Ready                                     2m      reference Project config-control/audit-p
config-con  IAMPolicyMember/log-reader                Skipped       InProgress              Ready                                     87m     Update call failed: error setting policy
config-con  IAMPolicyMember/log-writer                Skipped       InProgress              Ready                                     87m     Update call failed: error setting policy
config-con  IAMPolicyMember/organization-viewer       Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMServiceAccount/billing-service-accoun  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  LoggingLogSink/audit-bucket-sink          Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/a
config-con  LoggingLogSink/logs-bucket-sink           Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/l
config-con  Folder/audit-and-security                 Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.audit           Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.security        Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/automation                         Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking.nonprod  Successful    InProgress              Ready                                     2m      Update in progress
config-con  Folder/infrastructure.networking.prodnet  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.sharedinfrastructu  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/sandbox                            Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/shared-services                    Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads                          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.dev                      Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.prod                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.uat                      Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Project/audit-prj-id-oldv1                Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/guardrails-project-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-host-prj-nonprod-oldv1        Successful    InProgress              Ready                                     2m      reference Folder config-control/infrastr
config-con  Project/net-host-prj-prod-oldv1           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-per-prj-common-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ResourceManagerPolicy/allowed-contact-do  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/allowed-policy-mem  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-guest-attr  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-nested-vir  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-serial-por  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-serviceacc  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-vpc-extern  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/require-shielded-v  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/require-trusted-im  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
NAMESPACE   RESOURCE                                  ACTION        STATUS      RECONCILED  CONDITIONS                                AGE     MESSAGE
            ConstraintTemplate/cloudmarketplaceconfi  Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/datalocation           Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/limitegresstraffic     Skipped       Current                 <None>                                    87m     Resource is current
            ConstraintTemplate/namingpolicy           Skipped       Current                 <None>                                    87m     Resource is current
config-con  ConfigMap/setters                         Skipped       Current                 <None>                                    88m     Resource is always ready
config-con  AccessContextManagerAccessLevel/commonac  Skipped       InProgress              Ready                                     88m     reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessLevel/nonprodp  Successful    InProgress              Ready                                     2m      reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessLevel/prodacce  Skipped       InProgress              Ready                                     88m     reference AccessContextManagerAccessPoli
config-con  AccessContextManagerAccessPolicy/orgacce  Skipped       InProgress              Ready                                     88m     Update call failed: error applying desir
config-con  ComputeFirewall/allow-egress-internet     Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pr  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pu  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress         Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress-pr      Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingressp        Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/computefirewall-sample-d  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/prod-firewall-default-de  Skipped       InProgress              Ready                                     88m     reference ComputeNetwork config-control/
config-con  ComputeNetwork/common-ha-perimeter        Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/common-mgmt-perimeter      Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/nonprod-sharedvpc          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/priv-perimeter             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/prod-sharedvpc             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/public-perimeter           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeProjectMetadata/nonprod-oslogin-m  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeRoute/egress-internet-nonprod      Skipped       InProgress              Ready                                     87m     reference ComputeNetwork config-control/
config-con  ComputeRoute/egress-internet-prod         Skipped       InProgress              Ready                                     87m     reference ComputeNetwork config-control/
config-con  ComputeSharedVPCHostProject/computeshare  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ComputeSharedVPCHostProject/nonprod-shar  Skipped       InProgress              Ready                                     87m     Update call failed: error applying desir
config-con  ComputeSubnetwork/common-ha-perimeter-su  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/management              Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/nonprod-sharedvpc-subn  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/priv-perimeter-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/prod-sharedvpc-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/public-perimeter-subne  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  IAMPartialPolicy/audit-sink-writer        Skipped       InProgress              Ready                                     87m     reference StorageBucket config-control/a
config-con  IAMPartialPolicy/log-sink-writer          Skipped       InProgress              Ready                                     87m     reference StorageBucket config-control/l
config-con  IAMPolicyMember/audit-viewer              Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMPolicyMember/billing-iam-member        Successful    InProgress              Ready                                     2m      reference Project config-control/audit-p
config-con  IAMPolicyMember/log-reader                Skipped       InProgress              Ready                                     87m     Update call failed: error setting policy
config-con  IAMPolicyMember/log-writer                Skipped       InProgress              Ready                                     87m     Update call failed: error setting policy
config-con  IAMPolicyMember/organization-viewer       Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMServiceAccount/billing-service-accoun  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  LoggingLogSink/audit-bucket-sink          Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/a
config-con  LoggingLogSink/logs-bucket-sink           Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/l
config-con  Folder/audit-and-security                 Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.audit           Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.security        Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/automation                         Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking.nonprod  Successful    InProgress              Ready                                     2m      Update in progress
config-con  Folder/infrastructure.networking.prodnet  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.sharedinfrastructu  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/sandbox                            Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/shared-services                    Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads                          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.dev                      Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.prod                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.uat                      Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Project/audit-prj-id-oldv1                Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/guardrails-project-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-host-prj-nonprod-oldv1        Successful    InProgress              Ready                                     2m      reference Folder config-control/infrastr
config-con  Project/net-host-prj-prod-oldv1           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-per-prj-common-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ResourceManagerPolicy/allowed-contact-do  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/allowed-policy-mem  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-guest-attr  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-nested-vir  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-serial-por  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-serviceacc  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/disable-vpc-extern  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/require-shielded-v  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/require-trusted-im  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/restrict-loadbalan  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/restrict-os-login   Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/restrict-resource-  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/restrict-sql-publi  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/restrict-vm-extern  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/restrict-vpc-lien-  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/restrict-vpc-peeri  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/skip-default-netwo  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/storage-public-acc  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/uniform-bucket-lev  Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  ResourceManagerPolicy/vm-can-ip-forward   Skipped       InProgress              Ready                                     87m     Update call failed: error fetching live
config-con  Service/common-nethost-service-compute    Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/common-nethost-service-logging    Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/nonprod-nethost-service-compute   Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/nonprod-nethost-service-dns       Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/nonprod-nethost-service-logging   Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/prod-nethost-service-compute      Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/prod-nethost-service-logging      Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  StorageBucket/audit-audit-prj-id-oldv1    Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  StorageBucket/log-bucket-audit-prj-id-ol  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir

michael@cloudshell:~/dev/pdt-oldev/obriensystems (controller-oldev-3495)$
obriensystems commented 1 year ago

delete cluster - restart after kpt live destroy

 ./deployment.sh -b pdt-oldev -u pdtoldev -c false -l false -d true -p controller-oldev-3495

 cp pubsec-declarative-toolkit/solutions/landing-zone/setters.yaml landing-zone/ 
michael@cloudshell:~/dev/pdt-oldev/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pdt-oldev)$  ./deployment.sh -b pdt-oldev -u pdtoldev -c true -l true -d false
Date: Wed 07 Dec 2022 09:42:15 PM UTC
Timestamp: 1670449335
running with: -b pdt-oldev -u pdtoldev -c true -l true -d false -p
Updated property [core/project].
Switched back to boot project pdt-oldev
Start: 1670449336
unique string: pdtoldev
REGION: northamerica-northeast1
NETWORK: pdt-pdtoldev-vpc
SUBNET: pdt-pdtoldev-sn
CLUSTER: pdt-pdtoldev
CC_PROJECT_ID:
BOOT_PROJECT_ID: pdt-oldev
BILLING_ID: 011D7E-BD499C-CF71C5
ORG_ID: 583675367868
obriensystems commented 1 year ago 

526  gcloud config set project kcc-lz-8597
  527  ls
  528  gcloud anthos config controller get-credentials $CLUSTER  --location $REGION
  529  kpt pkg get https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit.git/solutions/landing-zone landing-zone
  530  cp pubsec-declarative-toolkit/solutions/landing-zone/setters.yaml landing-zone/
  531  cp pubsec-declarative-toolkit/solutions/landing-zone/.krmignore landing-zone/
  532  kpt live init landing-zone --namespace config-control
  533  kpt fn render landing-zone
  534  kpt live apply landing-zone --reconcile-timeout=2m --output=table
AMESPACE   RESOURCE                                  ACTION        STATUS      RECONCILED  CONDITIONS                                AGE     MESSAGE
            ConstraintTemplate/cloudmarketplaceconfi  Successful    Current                 <None>                                    2m      Resource is current
            ConstraintTemplate/datalocation           Successful    Current                 <None>                                    2m      Resource is current
            ConstraintTemplate/limitegresstraffic     Successful    Current                 <None>                                    2m      Resource is current
            ConstraintTemplate/namingpolicy           Successful    Current                 <None>                                    2m      Resource is current
config-con  ConfigMap/setters                         Successful    Current                 <None>                                    3m      Resource is always ready
config-con  AccessContextManagerAccessLevel/commonac  Successful    Current                 Ready                                     3m      Resource is Ready
config-con  AccessContextManagerAccessLevel/nonprodp  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  AccessContextManagerAccessLevel/prodacce  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  AccessContextManagerAccessPolicy/orgacce  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ComputeFirewall/allow-egress-internet     Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pr  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-egress-internet-pu  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress         Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingress-pr      Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/allow-ssh-ingressp        Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/computefirewall-sample-d  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeFirewall/prod-firewall-default-de  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeNetwork/common-ha-perimeter        Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/common-mgmt-perimeter      Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/nonprod-sharedvpc          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/priv-perimeter             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/prod-sharedvpc             Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeNetwork/public-perimeter           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeProjectMetadata/nonprod-oslogin-m  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeRoute/egress-internet-nonprod      Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeRoute/egress-internet-prod         Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSharedVPCHostProject/computeshare  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeSharedVPCHostProject/nonprod-shar  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ComputeSubnetwork/common-ha-perimeter-su  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/management              Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/nonprod-sharedvpc-subn  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/priv-perimeter-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/prod-sharedvpc-subnet   Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  ComputeSubnetwork/public-perimeter-subne  Successful    InProgress              Ready                                     2m      reference ComputeNetwork config-control/
config-con  IAMPartialPolicy/audit-sink-writer        Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/a
config-con  IAMPartialPolicy/log-sink-writer          Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/l
config-con  IAMPolicyMember/audit-viewer              Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMPolicyMember/billing-iam-member        Successful    InProgress              Ready                                     2m      reference Project config-control/audit-p
config-con  IAMPolicyMember/log-reader                Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMPolicyMember/log-writer                Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMPolicyMember/organization-viewer       Successful    Current                 Ready                                     2m      Resource is Ready
config-con  IAMServiceAccount/billing-service-accoun  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  LoggingLogSink/audit-bucket-sink          Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/a
config-con  LoggingLogSink/logs-bucket-sink           Successful    InProgress              Ready                                     2m      reference StorageBucket config-control/l
config-con  Folder/audit-and-security                 Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.audit           Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/audit-and-security.security        Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/automation                         Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking.nonprod  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.networking.prodnet  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/infrastructure.sharedinfrastructu  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/sandbox                            Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/shared-services                    Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads                          Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.dev                      Successful    InProgress              Ready                                     2m      Update in progress
config-con  Folder/workloads.prod                     Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Folder/workloads.uat                      Successful    InProgress              Ready                                     2m      Update in progress
config-con  Project/audit-prj-id-oldv1                Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/guardrails-project-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-host-prj-nonprod-oldv1        Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-host-prj-prod-oldv1           Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  Project/net-per-prj-common-oldv1          Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ResourceManagerPolicy/allowed-contact-do  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/allowed-policy-mem  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/disable-guest-attr  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/disable-nested-vir  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/disable-serial-por  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/disable-serviceacc  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/disable-vpc-extern  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/require-shielded-v  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/require-trusted-im  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/restrict-loadbalan  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/restrict-os-login   Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/restrict-resource-  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ResourceManagerPolicy/restrict-sql-publi  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/restrict-vm-extern  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/restrict-vpc-lien-  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/restrict-vpc-peeri  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/skip-default-netwo  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/storage-public-acc  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  ResourceManagerPolicy/uniform-bucket-lev  Successful    Current                 Ready                                     2m      Resource is Ready
config-con  ResourceManagerPolicy/vm-can-ip-forward   Successful    Current                 Ready                                     2m      Resource is Ready
config-con  Service/common-nethost-service-compute    Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/common-nethost-service-logging    Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/nonprod-nethost-service-compute   Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/nonprod-nethost-service-dns       Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/nonprod-nethost-service-logging   Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/prod-nethost-service-compute      Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  Service/prod-nethost-service-logging      Successful    InProgress              Ready                                     2m      Update call failed: error fetching live
config-con  StorageBucket/audit-audit-prj-id-oldv1    Successful    InProgress              Ready                                     2m      Update call failed: error applying desir
config-con  StorageBucket/log-bucket-audit-prj-id-ol  Successful    InProgress              Ready                                     2m      Update call failed: error applying desir

michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$

michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$ kubectl get gcp
NAME                                                                                                 AGE    READY   STATUS     STATUS AGE
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/commonaccesslevels        6m4s   True    UpToDate   5m59s
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/nonprodperimaccesslevel   6m3s   True    UpToDate   6m
accesscontextmanageraccesslevel.accesscontextmanager.cnrm.cloud.google.com/prodaccesslevels          6m3s   True    UpToDate   6m

NAME                                                                                          AGE    READY   STATUS     STATUS AGE
accesscontextmanageraccesspolicy.accesscontextmanager.cnrm.cloud.google.com/orgaccesspolicy   6m3s   True    UpToDate   6m1s

NAME                                                                        AGE     READY   STATUS         STATUS AGE
computeprojectmetadata.compute.cnrm.cloud.google.com/nonprod-oslogin-meta   5m59s   False   UpdateFailed   5m59s

NAME                                                                         AGE     READY   STATUS               STATUS AGE
computesubnetwork.compute.cnrm.cloud.google.com/common-ha-perimeter-subnet   5m57s   False   DependencyNotReady   5m57s
computesubnetwork.compute.cnrm.cloud.google.com/management                   5m57s   False   DependencyNotReady   5m57s
computesubnetwork.compute.cnrm.cloud.google.com/nonprod-sharedvpc-subnet     5m57s   False   DependencyNotReady   5m57s
computesubnetwork.compute.cnrm.cloud.google.com/priv-perimeter-subnet        5m56s   False   DependencyNotReady   5m56s
computesubnetwork.compute.cnrm.cloud.google.com/prod-sharedvpc-subnet        5m56s   False   DependencyNotReady   5m56s
computesubnetwork.compute.cnrm.cloud.google.com/public-perimeter-subnet      5m56s   False   DependencyNotReady   5m56s

NAME                                                                                           AGE     READY   STATUS         STATUS AGE
computesharedvpchostproject.compute.cnrm.cloud.google.com/computesharedvpchostproject-sample   5m58s   False   UpdateFailed   5m58s
computesharedvpchostproject.compute.cnrm.cloud.google.com/nonprod-shared-vpc-host              5m58s   False   UpdateFailed   5m58s

NAME                                                                 AGE    READY   STATUS         STATUS AGE
computenetwork.compute.cnrm.cloud.google.com/common-ha-perimeter     6m2s   False   UpdateFailed   6m2s
computenetwork.compute.cnrm.cloud.google.com/common-mgmt-perimeter   6m2s   False   UpdateFailed   6m1s
computenetwork.compute.cnrm.cloud.google.com/nonprod-sharedvpc       6m2s   False   UpdateFailed   6m1s
computenetwork.compute.cnrm.cloud.google.com/priv-perimeter          6m1s   False   UpdateFailed   6m1s
computenetwork.compute.cnrm.cloud.google.com/prod-sharedvpc          6m1s   False   UpdateFailed   6m1s
computenetwork.compute.cnrm.cloud.google.com/public-perimeter        6m     False   UpdateFailed   6m

NAME                                                                        AGE    READY   STATUS               STATUS AGE
computefirewall.compute.cnrm.cloud.google.com/allow-egress-internet         6m4s   False   DependencyNotReady   6m4s
computefirewall.compute.cnrm.cloud.google.com/allow-egress-internet-pr      6m4s   False   DependencyNotFound   6m4s
computefirewall.compute.cnrm.cloud.google.com/allow-egress-internet-pu      6m4s   False   DependencyNotReady   6m4s
computefirewall.compute.cnrm.cloud.google.com/allow-ssh-ingress             6m4s   False   DependencyNotFound   6m4s
computefirewall.compute.cnrm.cloud.google.com/allow-ssh-ingress-pr          6m3s   False   DependencyNotFound   6m3s
computefirewall.compute.cnrm.cloud.google.com/allow-ssh-ingressp            6m3s   False   DependencyNotFound   6m3s
computefirewall.compute.cnrm.cloud.google.com/computefirewall-sample-deny   6m3s   False   DependencyNotFound   6m3s
computefirewall.compute.cnrm.cloud.google.com/prod-firewall-default-deny    6m2s   False   DependencyNotFound   6m2s

NAME                                                                 AGE   READY   STATUS               STATUS AGE
computeroute.compute.cnrm.cloud.google.com/egress-internet-nonprod   6m    False   DependencyNotReady   6m
computeroute.compute.cnrm.cloud.google.com/egress-internet-prod      6m    False   DependencyNotReady   6m

NAME                                                           AGE     READY   STATUS               STATUS AGE
iampartialpolicy.iam.cnrm.cloud.google.com/audit-sink-writer   5m58s   False   DependencyNotReady   5m58s
iampartialpolicy.iam.cnrm.cloud.google.com/log-sink-writer     5m58s   False   DependencyNotReady   5m58s

NAME                                                                  AGE     READY   STATUS         STATUS AGE
iamserviceaccount.iam.cnrm.cloud.google.com/billing-service-account   5m56s   False   UpdateFailed   5m56s

NAME                                                            AGE     READY   STATUS               STATUS AGE
iampolicymember.iam.cnrm.cloud.google.com/audit-viewer          5m58s   True    UpToDate             5m14s
iampolicymember.iam.cnrm.cloud.google.com/billing-iam-member    5m58s   False   DependencyNotReady   5m58s
iampolicymember.iam.cnrm.cloud.google.com/log-reader            5m57s   True    UpToDate             5m14s
iampolicymember.iam.cnrm.cloud.google.com/log-writer            5m57s   True    UpToDate             5m8s
iampolicymember.iam.cnrm.cloud.google.com/organization-viewer   5m57s   True    UpToDate             5m52s

NAME                                                             AGE     READY   STATUS               STATUS AGE
logginglogsink.logging.cnrm.cloud.google.com/audit-bucket-sink   5m57s   False   DependencyNotFound   5m57s
logginglogsink.logging.cnrm.cloud.google.com/logs-bucket-sink    5m57s   False   DependencyNotFound   5m57s

NAME                                                                                       AGE     READY   STATUS     STATUS AGE
folder.resourcemanager.cnrm.cloud.google.com/audit-and-security                            5m57s   True    UpToDate   5m46s
folder.resourcemanager.cnrm.cloud.google.com/audit-and-security.audit                      5m57s   True    UpToDate   5m34s
folder.resourcemanager.cnrm.cloud.google.com/audit-and-security.security                   5m57s   True    UpToDate   5m34s
folder.resourcemanager.cnrm.cloud.google.com/automation                                    5m57s   True    UpToDate   5m45s
folder.resourcemanager.cnrm.cloud.google.com/infrastructure                                5m56s   True    UpToDate   5m45s
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.networking                     5m56s   True    UpToDate   5m27s
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.networking.nonprodnetworking   5m56s   True    UpToDate   4m32s
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.networking.prodnetworking      5m55s   True    UpToDate   4m31s
folder.resourcemanager.cnrm.cloud.google.com/infrastructure.sharedinfrastructure           5m55s   True    UpToDate   5m33s
folder.resourcemanager.cnrm.cloud.google.com/sandbox                                       5m55s   True    UpToDate   5m43s
folder.resourcemanager.cnrm.cloud.google.com/shared-services                               5m54s   True    UpToDate   5m43s
folder.resourcemanager.cnrm.cloud.google.com/workloads                                     5m54s   True    UpToDate   5m43s
folder.resourcemanager.cnrm.cloud.google.com/workloads.dev                                 5m54s   True    UpToDate   3m35s
folder.resourcemanager.cnrm.cloud.google.com/workloads.prod                                5m54s   True    UpToDate   4m20s
folder.resourcemanager.cnrm.cloud.google.com/workloads.uat                                 5m53s   True    UpToDate   3m35s

NAME                                                                                               AGE     READY   STATUS         STATUS AGE
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-contact-domains                5m51s   True    UpToDate       5m50s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/allowed-policy-member-domain           5m51s   True    UpToDate       5m50s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-guest-attribute-access         5m51s   True    UpToDate       5m51s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-nested-virtualization          5m51s   True    UpToDate       5m50s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-serial-port-access             5m50s   True    UpToDate       5m50s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-serviceaccount-key-creation    5m50s   True    UpToDate       5m49s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/disable-vpc-external-ipv6              5m50s   True    UpToDate       5m49s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/require-shielded-vm                    5m49s   True    UpToDate       5m48s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/require-trusted-images                 5m49s   True    UpToDate       5m48s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-loadbalancer-creation-types   5m49s   True    UpToDate       5m48s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-os-login                      5m48s   True    UpToDate       5m48s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-resource-locations            5m48s   False   UpdateFailed   5m48s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-sql-public-ip                 5m48s   True    UpToDate       5m47s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-vm-external-access            5m48s   True    UpToDate       5m47s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-vpc-lien-removal              5m47s   True    UpToDate       5m47s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-vpc-peering                   5m47s   True    UpToDate       5m46s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/skip-default-network-creation          5m47s   True    UpToDate       5m46s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/storage-public-access-prevention       5m46s   False   UpdateFailed   5m46s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/uniform-bucket-level-access            5m45s   True    UpToDate       5m45s
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/vm-can-ip-forward                      5m45s   True    UpToDate       5m44s

NAME                                                                       AGE     READY   STATUS         STATUS AGE
project.resourcemanager.cnrm.cloud.google.com/audit-prj-id-oldv1           5m53s   False   UpdateFailed   5m53s
project.resourcemanager.cnrm.cloud.google.com/guardrails-project-oldv1     5m53s   False   UpdateFailed   5m53s
project.resourcemanager.cnrm.cloud.google.com/net-host-prj-nonprod-oldv1   5m52s   False   UpdateFailed   5m52s
project.resourcemanager.cnrm.cloud.google.com/net-host-prj-prod-oldv1      5m52s   False   UpdateFailed   5m52s
project.resourcemanager.cnrm.cloud.google.com/net-per-prj-common-oldv1     5m52s   False   UpdateFailed   5m52s

NAME                                                                         AGE     READY   STATUS         STATUS AGE
service.serviceusage.cnrm.cloud.google.com/common-nethost-service-compute    5m44s   False   UpdateFailed   5m41s
service.serviceusage.cnrm.cloud.google.com/common-nethost-service-logging    5m44s   False   UpdateFailed   5m40s
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-compute   5m44s   False   UpdateFailed   5m40s
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-dns       5m44s   False   UpdateFailed   5m40s
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-logging   5m43s   False   UpdateFailed   5m40s
service.serviceusage.cnrm.cloud.google.com/prod-nethost-service-compute      5m43s   False   UpdateFailed   5m39s
service.serviceusage.cnrm.cloud.google.com/prod-nethost-service-logging      5m43s   False   UpdateFailed   5m39s

NAME                                                                        AGE     READY   STATUS         STATUS AGE
storagebucket.storage.cnrm.cloud.google.com/audit-audit-prj-id-oldv1        5m43s   False   UpdateFailed   5m43s
storagebucket.storage.cnrm.cloud.google.com/log-bucket-audit-prj-id-oldv1   5m43s   False   UpdateFailed   5m43s

michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$ kubectl describe service.serviceusage.cnrm.cloud.google.com/common-nethost-service-compute

  Warning  UpdateFailed  2s  service-controller  Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Project Service projects/net-per-prj-common-oldv1/services/: Request `List Project Services net-per-prj-common-oldv1` returned error: Batch request and retried single request "List Project Services net-per-prj-common-oldv1" both failed. Final error: Failed to list enabled services for project net-per-prj-common-oldv1: googleapi: Error 403: Project 'net-per-prj-common-oldv1' not found or permission denied.
obriensystems commented 1 year ago

Issue is SA is not a BAA on the billing project - will need to pre-create the SA

michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$ kubectl describe project.resourcemanager.cnrm.cloud.google.com/audit-prj-id-oldv1

 Warning  UpdateFailed        82s (x13 over 15m)  project-controller  Update call failed: error applying desired state: summary: failed pre-requisites: missing permission on "billingAccounts/011D7E-BD499C-CF71C5": billing.resourceAssociations.creat

kubectl get ConfigConnectorContext -n config-control -o jsonpath='{.items[0].spec.googleServiceAccount}'
service-946263025335@gcp-sa-yakima.iam.gserviceaccount.com
Screen Shot 2022-12-07 at 20 23 01

after applying, wait 60 sec then do another kpt render - 26 failures down to 22

michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$ kpt live apply landing-zone --reconcile-timeout=2m --output=table

michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$ kubectl get gcp | grep UpdateFailed | wc -l
22

michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$ kubectl get gcp | grep UpdateFailed 
computeprojectmetadata.compute.cnrm.cloud.google.com/nonprod-oslogin-meta   34m   False   UpdateFailed   34m
computesharedvpchostproject.compute.cnrm.cloud.google.com/computesharedvpchostproject-sample   34m   False   UpdateFailed   34m
computesharedvpchostproject.compute.cnrm.cloud.google.com/nonprod-shared-vpc-host              34m   False   UpdateFailed   34m
computenetwork.compute.cnrm.cloud.google.com/common-ha-perimeter     34m   False   UpdateFailed   34m
computenetwork.compute.cnrm.cloud.google.com/common-mgmt-perimeter   34m   False   UpdateFailed   34m
computenetwork.compute.cnrm.cloud.google.com/nonprod-sharedvpc       34m   False   UpdateFailed   34m
computenetwork.compute.cnrm.cloud.google.com/priv-perimeter          34m   False   UpdateFailed   34m
computenetwork.compute.cnrm.cloud.google.com/prod-sharedvpc          34m   False   UpdateFailed   34m
computenetwork.compute.cnrm.cloud.google.com/public-perimeter        34m   False   UpdateFailed   34m
project.resourcemanager.cnrm.cloud.google.com/audit-prj-id-oldv1           33m   False   UpdateFailed   33m
project.resourcemanager.cnrm.cloud.google.com/guardrails-project-oldv1     33m   False   UpdateFailed   33m
project.resourcemanager.cnrm.cloud.google.com/net-host-prj-nonprod-oldv1   33m   False   UpdateFailed   33m
project.resourcemanager.cnrm.cloud.google.com/net-host-prj-prod-oldv1      33m   False   UpdateFailed   33m
project.resourcemanager.cnrm.cloud.google.com/net-per-prj-common-oldv1     33m   False   UpdateFailed   33m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/restrict-resource-locations            33m   False   UpdateFailed   33m
resourcemanagerpolicy.resourcemanager.cnrm.cloud.google.com/storage-public-access-prevention       33m   False   UpdateFailed   33m
service.serviceusage.cnrm.cloud.google.com/common-nethost-service-compute    33m   False   UpdateFailed   33m
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-compute   33m   False   UpdateFailed   33m
service.serviceusage.cnrm.cloud.google.com/nonprod-nethost-service-dns       33m   False   UpdateFailed   33m
storagebucket.storage.cnrm.cloud.google.com/audit-audit-prj-id-oldv1        33m   False   UpdateFailed   33m
storagebucket.storage.cnrm.cloud.google.com/log-bucket-audit-prj-id-oldv1   33m   False   UpdateFailed   33m
michael@cloudshell:~/dev/pdt-oldev/obriensystems (kcc-lz-8597)$

actually error is now a quota failure - I need more than the current 15

    "@type": "type.googleapis.com/google.rpc.QuotaFailure",
    "violations": [
      {
        "description": "Cloud billing quota exceeded: https://support.google.com/code/contact/billing_quota_increase",
        "subject": "billingAccounts/011D7E-BD499C-CF71C5"
      }
obriensystems commented 1 year ago

kpt live delete landing-zone adding docs in https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/issues/209

obriensystems commented 1 year ago

Need to delete SA created

obriensystems commented 1 year ago

Cloud Functions requirements

fmichaelobrien commented 1 year ago

restarting dev cluster on alt..g.z

root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$ git pull
./deployment.sh -b pubsec-declarative-agz -u pdt1 -c true -l false -d false
root_@cloudshell:~/wse_github/obriensystems/pubsec-declarative-toolkit/solutions/landing-zone (pubsec-declarative-agz)$ ./deployment.sh -b pubsec-declarative-agz -u pdt1 -c true -l false -d false -p config-controller
Date: Fri 13 Jan 2023 07:20:07 PM UTC
Timestamp: 1673637607
running with: -b pubsec-declarative-agz -u pdt1 -c true -l false -d false -p config-controller
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
Start: 1673637608
unique string: pdt1
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
SUBNET: pdt-pdt1-sn
CLUSTER: pdt-pdt1
CC_PROJECT_ID: config-controller-849
BOOT_PROJECT_ID: pubsec-declarative-agz
BILLING_ID: 019...76
ORGID: 6..2
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/config-controller-849].
Waiting for [operations/cp.7009910468324531381] to finish...done.    
Enabling service [cloudapis.googleapis.com] on project [config-controller-849]...
Operation "operations/acat.p2-788133329726-e293d33f-ee7f-457b-86ce-7911c53fea10" finished successfully.
Updated property [core/project] to [config-controller-849].
Created KCC project: config-controller-849
Updated property [core/project].
billingAccountName: billingAccounts/019283-6F1AB5-7AD576
billingEnabled: true
name: projects/config-controller-849/billingInfo
projectId: config-controller-849
Enabling APIs
Operation "operations/acf.p2-788133329726-61b39d1b-bec7-4e1b-b316-8929736ac27e" finished successfully.
Create VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/config-controller-849/global/networks/pdt-pdt1-vpc].
NAME: pdt-pdt1-vpc
SUBNET_MODE: CUSTOM
BGP_ROUTING_MODE: REGIONAL
IPV4_RANGE:
GATEWAY_IPV4:

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network pdt-pdt1-vpc --allow tcp:22,tcp:3389,icmp

Create subnet pdt-pdt1-sn off VPC: pdt-pdt1-vpc
Created [https://www.googleapis.com/compute/v1/projects/config-controller-849/regions/northamerica-northeast1/subnetworks/pdt-pdt1-sn].
NAME: pdt-pdt1-sn
REGION: northamerica-northeast1
NETWORK: pdt-pdt1-vpc
RANGE: 192.168.0.0/16
STACK_TYPE: IPV4_ONLY
IPV6_ACCESS_TYPE:
INTERNAL_IPV6_PREFIX:
EXTERNAL_IPV6_PREFIX:
Creating Anthos KCC autopilot cluster pdt-pdt1 in region northamerica-northeast1 in subnet pdt-pdt1-sn off VPC pdt-pdt1-vpc
Create request issued for: [pdt-pdt1]
Waiting for operation [projects/config-controller-849/locations/northamerica-northeast1/operations/operation-1673637714446-5f22a2573c85a-40bdc228-e670a5cf] to complete...working.. 

aiting for operation [projects/config-controller-849/locations/northamerica-northeast1/operations/operation-1673637714446-5f22a2573c85a-40bdc228-e670a5cf] to complete...done.     
Created instance [pdt-pdt1].
Fetching cluster endpoint and auth data.
kubeconfig entry generated for krmapihost-pdt-pdt1.
Cluster create time: 1107 sec
Fetching cluster endpoint and auth data.
kubeconfig entry generated for krmapihost-pdt-pdt1.
Context "gke_config-controller-849_northamerica-northeast1_krmapihost-pdt-pdt1" modified.
Active namespace is "config-control".
List Clusters:
NAME: pdt-pdt1
LOCATION: northamerica-northeast1
STATE: RUNNING
Total Duration: 1217 sec
Date: Fri 13 Jan 2023 07:40:25 PM UTC
Timestamp: 1673638825
Updated property [core/project].
Switched back to boot project pubsec-declarative-agz
**** Done ****