Package unusable with k8s deployment+ingress+tls termination

[alee-x]

Describe the bug This package cannot be used when Zeebe Gateway is deployed on Kubernetes, exposed via an Ingress secured with TLS, with TLS terminating at the Ingress and not the pod.

When using a K8s Ingress with GRPC protocol, http2 can only be served over port 443 (see this issue or this issue), meaning that the Ingress has to use TLS. The generally accepted solution in this case is that you then terminate TLS at the Ingress and allow plain text communication between the pod and the ingress controller.

This pattern would result in a Zeebe Gateway URL that looks like https://zeebe-gateway-grpc.my.ingress.controller, but the fact we're terminating TLS at the edge means the client needs to be instantiated with .UsePlainText().

However, two pieces of behaviour stop this from working:

1. The ZeebePlainClientBuilder does not allow plain clients to be created when using a https protocol on the Zeebe URL.
2. The ZeebeClient always assumes that there are credentials when using https.

The workaround is to expose the Zeebe Gateway using an external Load Balancer so it gets an IP, however in cloud deployments (AWS, AKS, etc) ELBs are much more expensive than Ingress controllers and are generally more hassle.

To Reproduce Steps to reproduce the behavior:

1. Deploy the camunda-platform helm chart on a K8s cluster with an ingress controller, in the values.yaml file you'll need to add:

   zeebeGateway:
   ingress:
   grpc:
     enabled: true
     className: "YOUR NGINX CONTROLLER CLASS NAME HERE"
     host: "grpc-zeebe-gateway.your.domain.address"
     annotations:
       ingress.kubernetes.io/rewrite-target: "/"
       nginx.ingress.kubernetes.io/ssl-redirect: "false"
       nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
     tls:
       enabled: true
       secretName: camunda-platform-zeebe-gateway-grpc

   (this assumes that you have already created a tls secret with correct certs and keys called camunda-platform-zeebe-gateway-grpc)

2. Using the Client.Examples, change the ZeebeUrl to https://<whatever you just put as the ingress host> and leave .UsePlainText().
3. Watch it fail.

Expected behavior I should be able to configure the client to use both a https URL and plaintext for cases where TLS is terminated at the edge.

Enviroment (please complete the following information):

• Kubernetes version: v1.28.2+k3s1
• ingress-nginx: 4.10.1
• this client version: 2.6.0

[xlegalles]

Just by curiosity: why do you expose zeebe through the ingress? We also use it in a k8s cluster but we have not this problem because we don't expose zeebe which is restricted to an internal usage.