GoogleCloudPlatform / cloud-sql-proxy-operator

A Kubernetes Operator to automatically configure secure connections to Cloud SQL
Apache License 2.0
99 stars 11 forks source link

InitContainer Ordering Issue with CloudSQL Operator 1.6.0 in Istio-Managed Environments #641

Open utmaks opened 3 days ago

utmaks commented 3 days ago

Expected Behavior

Right order for initContainers / integration with Istio.

Actual Behavior

Description: During the upgrade of CloudSQL Operator to version 1.6.0, issues were encountered when deploying new instances of the application. While the existing application instances continued functioning as expected, attempts to start new pods failed due to critical container startup errors, which would be unacceptable in a production environment.

Details: After upgrading to version 1.6.0 to utilize the minSigtermDelay parameter, the following problem was observed:

The issue seems tied to Istio sidecar injection. Specifically:

Resolution Attempt: Disabling Istio temporarily resolved the problem, allowing the CloudSQL container to start correctly in the new pods. However, this is not an acceptable solution for production workloads where Istio is required.

Proposed Fix for CloudSQL Operator team: Reintroduce user-configurable options for sidecar compatibility (e.g., sidecarType), as seen in earlier commits, to prevent such failures in Istio environments and ensure robust behavior during deployment in production scenarios.

Steps to Reproduce the Problem

  1. Install Istio (1.19.3)
  2. Install Operator
  3. Create a Deployment with CloudSQL annotation in namespace managed by Istio

Specifications

jackwotherspoon commented 3 days ago

Thanks for this @utmaks 👏

@hessjcg is OOO this week but will take a look next week when he is back