This PR introduces support for session affinity settings in the Kubernetes deployment of the AI service. As the AI service is stateful, scaling replicas can lead to increased errors for "query not found" if results are retrieved from the wrong pod. To address this, we are using Istio to manage session affinity and improve reliability.
How to validate?
Scale up the replicas of the service on the Kubernetes cluster:
This PR introduces support for session affinity settings in the Kubernetes deployment of the AI service. As the AI service is stateful, scaling replicas can lead to increased errors for "query not found" if results are retrieved from the wrong pod. To address this, we are using Istio to manage session affinity and improve reliability.
How to validate?
Scale up the replicas of the service on the Kubernetes cluster:
Create a test pod:
Access the test pod:
Validate the health check endpoint with a header and trace the result in pod log:
See Also