While destroying a GCP PKS deployment with three service account (opsman, pks-worker-node, pks-master-node), we found we had to run leftovers v0.48.0 three times to delete all of the service accounts. Only one service account was deleted per run.
The errors we received were:
IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-opsman@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Deleting...
[IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-pks-worker-node@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Deleting...
[IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-pks-master-node@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Deleting...
[IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-opsman@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted
[IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-pks-master-node@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted
[IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-pks-worker-node@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Deleted!
[DNS Managed Zone: grape-staging-zone] Deleting...
[DNS Managed Zone: grape-staging-zone] Deleted!
2 errors occurred:
* [IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-opsman@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted
* [IAM Service Account: projects/pcf-toolsmiths-dev-1/serviceAccounts/grape-staging-pks-master-node@pcf-toolsmiths-dev-1.iam.gserviceaccount.com] Remove IAM Policy Bindings: Set Project IAM Policy: googleapi: Error 409: There were concurrent policy changes. Please retry the whole read-modify-write with exponential backoff., aborted
Would it be possible for leftovers to add retry logic in the case of a 409?
While destroying a GCP PKS deployment with three service account (opsman, pks-worker-node, pks-master-node), we found we had to run leftovers v0.48.0 three times to delete all of the service accounts. Only one service account was deleted per run.
The errors we received were:
Would it be possible for leftovers to add retry logic in the case of a 409?