I have at least one instance of an instance for which deprovision failed but where the instance was finalized and fully deleted. When this happens, the instance should remain in a failed state so that the user is aware that deprovision failed and can work with the broker operator to ensure the resources are deleted and that they are not being charged for an instance for which deprovision failed. The final remediation for this from a k8s standpoint is that the user or an operator can force-delete the ServiceInstance when they have worked through ensuring that the resources are actually deleted and they're no longer being charged.
Logs:
```
I1018 20:37:32.409486 63 controller.go:312] Creating client for ClusterServiceBroker ansible-service-broker, URL: https://asb.ansible-service-broker.svc:1338/ansible-service-broker/
I1018 20:37:32.409967 63 controller_instance.go:1471] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Setting condition "Ready" to False
I1018 20:37:32.409994 63 controller_instance.go:1490] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Found status change, condition "Ready": "True" -> "False"; setting lastTransitionTime to 2017-10-18 20:37:32.409958514 +0000 UTC
I1018 20:37:32.410014 63 controller_instance.go:1521] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Updating status
I1018 20:37:32.419723 63 controller_instance.go:292] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Deprovisioning
I1018 20:37:32.452711 63 controller_instance.go:377] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Received asynchronous de-provisioning response, 97cf1893d97f32dd339d1f7cba6b31bf at ansible-service-broker: response: &{Async:true OperationKey:0xc42045d9b0}
I1018 20:37:32.452764 63 controller_instance.go:1471] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Setting condition "Ready" to False
I1018 20:37:32.452783 63 controller_instance.go:1521] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Updating status
I1018 20:37:32.459836 63 event.go:218] Event(v1.ObjectReference{Kind:"ServiceInstance", Namespace:"fail-project", Name:"apb-push-fail-apb-pvrnz", UID:"0fd8f140-b444-11e7-adf7-0242ac110005", APIVersion:"servicecatalog.k8s.io", ResourceVersion:"91", FieldPath:""}): type: 'Normal' reason: 'Deprovisioning' The instance is being deprovisioned asynchronously
I1018 20:37:32.607803 63 controller.go:312] Creating client for ClusterServiceBroker ansible-service-broker, URL: https://asb.ansible-service-broker.svc:1338/ansible-service-broker/
I1018 20:37:32.608082 63 controller_instance.go:292] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Deprovisioning
I1018 20:37:32.620834 63 controller_instance.go:377] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Received asynchronous de-provisioning response, 97cf1893d97f32dd339d1f7cba6b31bf at ansible-service-broker: response: &{Async:true OperationKey:}
I1018 20:37:32.620873 63 controller_instance.go:1471] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Setting condition "Ready" to False
I1018 20:37:32.620885 63 controller_instance.go:1521] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Updating status
E1018 20:37:32.624978 63 controller_instance.go:1524] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Failed to update status: Operation cannot be fulfilled on serviceinstances.servicecatalog.k8s.io "apb-push-fail-apb-pvrnz": the object has been modified; please apply your changes to the latest version and try again
I1018 20:37:32.625007 63 controller.go:232] Error syncing ServiceInstance fail-project/apb-push-fail-apb-pvrnz: Operation cannot be fulfilled on serviceinstances.servicecatalog.k8s.io "apb-push-fail-apb-pvrnz": the object has been modified; please apply your changes to the latest version and try again
I1018 20:37:32.630136 63 controller_instance.go:921] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Processing
I1018 20:37:32.632493 63 controller.go:312] Creating client for ClusterServiceBroker ansible-service-broker, URL: https://asb.ansible-service-broker.svc:1338/ansible-service-broker/
I1018 20:37:32.632727 63 controller_instance.go:1013] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Polling last operation
I1018 20:37:32.646438 63 controller_instance.go:1107] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Poll returned "in progress" : %!q(*string=)
I1018 20:37:32.646508 63 controller_instance.go:1193] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": last operation not completed (still in progress)
I1018 20:37:32.876445 63 leaderelection.go:199] successfully renewed lease kube-service-catalog/service-catalog-controller-manager
I1018 20:37:33.459932 63 controller_instance.go:921] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Processing
I1018 20:37:33.465160 63 controller.go:312] Creating client for ClusterServiceBroker ansible-service-broker, URL: https://asb.ansible-service-broker.svc:1338/ansible-service-broker/
I1018 20:37:33.465584 63 controller_instance.go:1013] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Polling last operation
I1018 20:37:33.478728 63 controller_instance.go:1107] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Poll returned "in progress" : %!q(*string=)
I1018 20:37:33.478793 63 controller_instance.go:1193] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": last operation not completed (still in progress)
I1018 20:37:34.898760 63 leaderelection.go:199] successfully renewed lease kube-service-catalog/service-catalog-controller-manager
I1018 20:37:36.912250 63 leaderelection.go:199] successfully renewed lease kube-service-catalog/service-catalog-controller-manager
I1018 20:37:37.479017 63 controller_instance.go:921] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Processing
I1018 20:37:37.483058 63 controller.go:312] Creating client for ClusterServiceBroker ansible-service-broker, URL: https://asb.ansible-service-broker.svc:1338/ansible-service-broker/
I1018 20:37:37.483470 63 controller_instance.go:1013] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Polling last operation
I1018 20:37:37.504218 63 controller_instance.go:1107] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Poll returned "in progress" : %!q(*string=)
I1018 20:37:37.504311 63 controller_instance.go:1193] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": last operation not completed (still in progress)
I1018 20:37:38.920729 63 leaderelection.go:199] successfully renewed lease kube-service-catalog/service-catalog-controller-manager
I1018 20:37:40.929379 63 leaderelection.go:199] successfully renewed lease kube-service-catalog/service-catalog-controller-manager
I1018 20:37:42.938438 63 leaderelection.go:199] successfully renewed lease kube-service-catalog/service-catalog-controller-manager
I1018 20:37:44.946954 63 leaderelection.go:199] successfully renewed lease kube-service-catalog/service-catalog-controller-manager
I1018 20:37:45.504527 63 controller_instance.go:921] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Processing
I1018 20:37:45.509046 63 controller.go:312] Creating client for ClusterServiceBroker ansible-service-broker, URL: https://asb.ansible-service-broker.svc:1338/ansible-service-broker/
I1018 20:37:45.509519 63 controller_instance.go:1013] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Polling last operation
I1018 20:37:45.544512 63 controller_instance.go:1107] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Poll returned "failed" : %!q(*string=)
I1018 20:37:45.544624 63 controller_instance.go:1471] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Setting condition "Ready" to Unknown
I1018 20:37:45.544648 63 controller_instance.go:1490] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Found status change, condition "Ready": "False" -> "Unknown"; setting lastTransitionTime to 2017-10-18 20:37:45.544618301 +0000 UTC
I1018 20:37:45.544680 63 controller_instance.go:1471] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Setting condition "Failed" to True
I1018 20:37:45.544691 63 controller_instance.go:1501] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Setting lastTransitionTime, condition "Failed" to 2017-10-18 20:37:45.544676883 +0000 UTC
I1018 20:37:45.544707 63 controller_instance.go:1521] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Updating status
I1018 20:37:45.544694 63 event.go:218] Event(v1.ObjectReference{Kind:"ServiceInstance", Namespace:"fail-project", Name:"apb-push-fail-apb-pvrnz", UID:"0fd8f140-b444-11e7-adf7-0242ac110005", APIVersion:"servicecatalog.k8s.io", ResourceVersion:"93", FieldPath:""}): type: 'Warning' reason: 'DeprovisionCallFailed' ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Error deprovisioning: ""
I1018 20:37:45.604743 63 controller_instance.go:217] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Clearing catalog finalizer
I1018 20:37:45.604792 63 controller_instance.go:1521] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Updating status
I1018 20:37:45.804806 63 controller_instance.go:70] ServiceInstance "fail-project/apb-push-fail-apb-pvrnz": Received dele
```
pmorie
commented
6 years ago
I have at least one instance of an instance for which deprovision failed but where the instance was finalized and fully deleted. When this happens, the instance should remain in a failed state so that the user is aware that deprovision failed and can work with the broker operator to ensure the resources are deleted and that they are not being charged for an instance for which deprovision failed. The final remediation for this from a k8s standpoint is that the user or an operator can force-delete the
ServiceInstancewhen they have worked through ensuring that the resources are actually deleted and they're no longer being charged.Logs: