feat: update/deletion strategy

[alexeldeib]

deletion is dangerous/expensive. we should be deliberate about whether we allow changes to immutable fields to trigger deletions or not.

e.g.

Create a valid resource group:

apiVersion: azure.alexeldeib.xyz/v1alpha1
kind: ResourceGroup
metadata:
  name: rg-sample
spec:
  strategy: delete
  subscriptionId: c69b07f1-f4da-401d-a2e3-6db35cc3d017
  name: ace-crd
  location: westus
status:
  provisioningState: succeeded
  reason: ''

Change location to a different, valid value:

apiVersion: azure.alexeldeib.xyz/v1alpha1
kind: ResourceGroup
metadata:
  name: rg-sample
spec:
  strategy: delete
  subscriptionId: 7c7833f6-24d9-4b55-b7ad-164da55c5e67
  name: ace-crd
  location: westus2
status:
  provisioningState: deleting
  reason: ''
.
.
.
(time passes)
.
.
.
apiVersion: azure.alexeldeib.xyz/v1alpha1
kind: ResourceGroup
metadata:
  name: rg-sample
spec:
  strategy: delete
  subscriptionId: 7c7833f6-24d9-4b55-b7ad-164da55c5e67
  name: ace-crd
  location: westus2
status:
  provisioningState: succeeded
  reason: ''

Expected behavior would be for controller to deletion westus resource group and eventually handle creation in westus2 (maybe with some lifetime timeout, like progressDeadlineSeconds)

another example:

Create a valid resource group:

apiVersion: azure.alexeldeib.xyz/v1alpha1
kind: ResourceGroup
metadata:
  name: rg-sample
spec:
  strategy: preserve
  subscriptionId: c69b07f1-f4da-401d-a2e3-6db35cc3d017
  name: ace-crd
  location: westus

Change location to a different, valid value:

apiVersion: azure.alexeldeib.xyz/v1alpha1
kind: ResourceGroup
metadata:
  name: rg-sample
spec:
  strategy: preserve
  subscriptionId: 7c7833f6-24d9-4b55-b7ad-164da55c5e67
  name: ace-crd
  location: westus2
status:
  provisioningState: failed
  reason: immutableFieldChanged

Expected behavior is (in probable order of preference):

• Webhook rejects user change.
• controller looks at actual state from Azure/stored state in status and determines this is a change to immutable field
  • does nothing, indicates error to user via logs, events, and status fields.
  • sends the operation to Azure, where it fails. current behavior!

[alexeldeib]

This is really about the user being explicit in what they want and having sane defaults. It applies to both update and delete.

Examples:

• Updating an immutable field with a strategy like preserve errors out.
• Updating an immutable field with a strategy like recreate performs deletion and recreation with downtime.
• Deleting an object with a deletion policy of "manual" (as a trivial additional example) might require a user to manually remove a finalizer. This might be useful for very valuable objects where deletion would be highly detrimental (think accidentally deleting a keyvault).