search

GoogleCloudPlatform / k8s-config-connector

GCP Config Connector, a Kubernetes add-on for managing GCP resources
https://cloud.google.com/config-connector/docs/overview
Apache License 2.0
898 stars 231 forks source link

Unable to disable deadLetterPolicy on PubsubSubscription #2606

Open dionborsboom opened 2 months ago

dionborsboom commented 2 months ago

Checklist

Bug Description

Disabling the deadLetterPolicy in a PubsubSubscription does not work. Removing the deadLetterPolicy parameter it from the spec makes it externally-managed instead of disabling it. The documentation describes deadLetterPolicy as If dead_letter_policy is not set, dead lettering is disabled. https://cloud.google.com/config-connector/docs/reference/resource-docs/pubsub/pubsubsubscription But this is only true if its never set in the first place. Disabling after having it enabled does not work.

I have tried this guide to see if one of the suggestions would disable it: https://cloud.google.com/config-connector/docs/troubleshooting#unable_to_remove_fields_from_resource_spec

I have tried adding an empty object like deadLetterPolicy: {} and deadLetterPolicy: null, but it does not work. The system will allow it, but ignores it and the status is always UpToDate.

I have also tried giving all parameter values in the deadLetterPolicy object "null" values as suggested in the troubleshooting doc, but the resource will return an error in the condition: 

status:
  conditions:
  - lastTransitionTime: "2024-09-02T13:22:29Z"
    message: 'Update call failed: error fetching live state: error converting resource
      config: error getting referenced resource from API server: resource reference
      is missing required ''name'' field'
    reason: UpdateFailed
    status: "False"
    type: Ready

Additional Diagnostic Information

none

Kubernetes Cluster Version

v1.28.11-gke.1019001

Config Connector Version

1.118.2

Config Connector Mode

namespaced mode (default)

Log Output

No response

Steps to reproduce the issue

  1. Create a PubsubSubscription with a deadLetterPolicy configured
  2. Remove deadLetterPolicy from the manifest / change the config to a "null" value

YAML snippets

---
# original subscription manifest
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  annotations:
    cnrm.cloud.google.com/management-conflict-prevention-policy: none
    cnrm.cloud.google.com/project-id: REDACTED
    cnrm.cloud.google.com/state-into-spec: absent
  creationTimestamp: "2024-09-02T13:15:12Z"
  finalizers:
  - cnrm.cloud.google.com/finalizer
  - cnrm.cloud.google.com/deletion-defender
  generation: 1
  name: my-sub
  namespace: REDACTED
spec:
  ackDeadlineSeconds: 300
  deadLetterPolicy:
    deadLetterTopicRef:
      name: my-dlt
      namespace: REDACTED
    maxDeliveryAttempts: 10
  enableMessageOrdering: false
  expirationPolicy:
    ttl: ""
  messageRetentionDuration: 604800s
  resourceID: my-sub
  retainAckedMessages: false
  retryPolicy:
    maximumBackoff: 600s
    minimumBackoff: 10s
  topicRef:
    name: my-topic
    namespace: REDACTED
status:
  conditions:
  - lastTransitionTime: "2024-09-02T13:15:14Z"
    message: The resource is up to date
    reason: UpToDate
    status: "True"
    type: Ready
observedGeneration: 1
---
# Modify to an empty object, does not disable the deadLetterPolicy in GCP
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  annotations:
    cnrm.cloud.google.com/management-conflict-prevention-policy: none
    cnrm.cloud.google.com/project-id: REDACTED
    cnrm.cloud.google.com/state-into-spec: absent
  creationTimestamp: "2024-09-02T13:15:12Z"
  finalizers:
  - cnrm.cloud.google.com/finalizer
  - cnrm.cloud.google.com/deletion-defender
  generation: 2
  name: my-sub
  namespace: REDACTED
spec:
  ackDeadlineSeconds: 300
  enableMessageOrdering: false
  deadLetterPolicy: {}
  expirationPolicy:
    ttl: ""
  messageRetentionDuration: 604800s
  resourceID: my-sub
  retainAckedMessages: false
  retryPolicy:
    maximumBackoff: 600s
    minimumBackoff: 10s
  topicRef:
    name: my-topic
    namespace: REDACTED
status:
  conditions:
  - lastTransitionTime: "2024-09-02T13:22:29Z"
    message: The resource is up to date
    reason: UpToDate
    status: "True"
    type: Ready
  observedGeneration: 2
---
# tried various iterations of null values as suggested in the troubleshooting docs
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  annotations:
    cnrm.cloud.google.com/management-conflict-prevention-policy: none
    cnrm.cloud.google.com/project-id: REDACTED
    cnrm.cloud.google.com/state-into-spec: absent
  creationTimestamp: "2024-09-02T13:15:12Z"
  finalizers:
  - cnrm.cloud.google.com/finalizer
  - cnrm.cloud.google.com/deletion-defender
  generation: 3
  name: my-sub
  namespace: REDACTED
spec:
  ackDeadlineSeconds: 300
  deadLetterPolicy:
    deadLetterTopicRef:
      name: ""
      namespace: ""
    maxDeliveryAttempts: 0
  enableMessageOrdering: false
  expirationPolicy:
    ttl: ""
  messageRetentionDuration: 604800s
  resourceID: my-sub
  retainAckedMessages: false
  retryPolicy:
    maximumBackoff: 600s
    minimumBackoff: 10s
  topicRef:
    name: my-topic
    namespace: REDACTED
status:
  conditions:
  - lastTransitionTime: "2024-09-02T13:34:01Z"
    message: 'Update call failed: error fetching live state: error converting resource
      config: error getting referenced resource from API server: resource reference
      is missing required ''name'' field'
    reason: UpdateFailed
    status: "False"
    type: Ready
  observedGeneration: 3
gemmahou commented 2 months ago

Currently remove an Object field is not fully supported by KCC:

For object type fields, currently in Config Connector there is no easy way to set a whole object type field as "NULL". You can try to set the subfields of the object type as empty or default following the guidance above and verify if it works.

To bypass the unsupported feature, we suggest that Cx abandon the KCC resource, leverage other tooling to disable/remove the Object type field (console, gcloud, etc.), and then acquire the resource by KCC.

Detailed steps:

-Update the YAML configuration of the Config Connector resource and set the cnrm.cloud.google.com/deletion-policy annotation to abandon. -Apply the updated YAML configuration to update the Config Connector resource's deletion policy and abandon the Config Connector resource. -Disable the field using other tooling(console, gcloud, etc). -Update the field that need to be disabled in the YAML configuration. -Apply the updated YAML configuration to acquire the abandoned resource.

Ref: https://cloud.google.com/config-connector/docs/troubleshooting#abandon_and_acquire_the_resource