BackendTlsPolicy specify multiple targetRefs of the same service, only one will work

[haorenfsa]

Description:

What issue is being seen? Describe what should be happening instead of the bug, for example: Envoy should not crash, the expected value isn't returned, etc.

When BackendTlsPolicy specify multiple targetRefs of the same service with different port, only the first one will work.

Repro steps:

Include sample requests, environment, etc. All data and inputs required to reproduce the bug.

Note: If there are privacy concerns, sanitize the data prior to sharing.

• start a service with 2 ports 8080 & 8081, and enable TLS with self-signed localhost certificate.

• create Gateway CR & 2 HTTPRoute CR for both ports.

• create ConfigMap my-ca with self-signed ca.crt.

• create BackendTLSPolicy

  apiVersion: gateway.networking.k8s.io/v1alpha3
  kind: BackendTLSPolicy
  metadata:
  name: my-service
  namespace: default
  spec:
  targetRefs:
  - group: ''
  kind: Service
  name: my-service
  sectionName: "8080"
  - group: ''
  kind: Service
  name: my-service
  sectionName: "8081"
  validation:
  caCertificateRefs:
  - name: my-ca
    group: ''
    kind: ConfigMap
  hostname: localhost

• curl gateway 8080 service with https, ok

• curl gateway 8081 service with https

• got following error:

  Client sent an HTTP request to an HTTPS server.

Environment:

Include the environment like gateway version, envoy version and so on.

gateway: v1.1.0

Logs:

Include the access logs and the Envoy logs.

[zhaohuabing]

Client sent an HTTP request to an HTTPS server.

@haorenfsa BackendTLSPolicy applies to the connection between the Envoy and the backend services. This error came from the connection between the client and the Envoy, so it has nothing to do with the BackendTLSPolicy.

Looks like the Gateway listener is HTTP protocol, so you got this error when sending an HTTP request to it.

Could you please paste the Gateway and HTTPRoute resources here?

[haorenfsa]

Em... Seems it has nothing todo with HTTPRoute or Gateway CR. I resolve it by removing the SectionName field.

apiVersion: gateway.networking.k8s.io/v1alpha3
kind: BackendTLSPolicy
metadata:
  name: my-service
  namespace: default
spec:
  targetRefs:
  - group: ''
    kind: Service
    name: my-service
-    sectionName: "8080"
-  - group: ''
-    kind: Service
-    name: my-service
-    sectionName: "8081"
  validation:
    caCertificateRefs:
    - name: my-ca
      group: ''
      kind: ConfigMap
    hostname: localhost

[haorenfsa]

By the way, the error comes from envoy to its backend. The link route is like:

client ---- https (SNI: myservice.com) ----> envoy ---- https (SNI: localhost) ----> backend

This happens when I'm using GCP L7 LoadBalancer with HTTP2/GRPC backend, and try migrate to L4 LoadBalancer with envoy gateway. GCP forces its backend to enable tls.

I created A Gateway with listener set to tls.mode= Terminate. Then I add HTTPRoute, and this BackendTLSPolicy.

[arkodg]

reopening the issue, because the config you specified, although verbose, is still valid, and should work

the other issue around upstream tls may be related to SNI and SAN, you may find the cert generation section in https://gateway.envoyproxy.io/docs/tasks/security/backend-tls/ useful