Example of envoy with HTTPS connection to grpc backend

hjfreyer commented 3 years ago

I cannot for the life of me figure out how to use Envoy to proxy grpc-web requests to a grpc backend over HTTPs.

My use case is that I'd like to use grpc-web with a service on GCP Run. GCP Run deployments are individual Docker containers, so to accomplish this, I have two options:

Put my grpc server and envoy into the same container. This is generally considered bad practice: containers should be single-purpose. I hit all sorts of sharp edges, and the general response is, "stop running multiple services in a single container."
Separate my grpc server from my envoy server. This is a tad heavyweight, but clearly the way Docker and GCP Run "want" me to do it.

The "problem" is that service-to-service communication between GCP Run deployments is over HTTPS. That's not a problem per se, but I simply can't make it work. I'd include the config I'm using, but I've tried so many things and I can't include them all. I've essentially been trying to combine the envoy.yaml in the Hello World example with this: https://farcaller.medium.com/how-to-configure-https-backends-in-envoy-b446727b2eb3.

The symptoms I'm seeing are:

When I use TLS, envoy generally doesn't send a response back to the client. The client just hangs forever.
Depending on the exact details of the configuration I'm using, I see either "503" or "404" in the envoy debug logs, and no further information besides the status code.

Establishing a secure connection to a grpc backend seems like it should be a supported configuration, so can we get an example of it in the docs?

stanley-cheung commented 3 years ago

Perhaps these examples may help?

https://github.com/salrashid123/grpc_web_with_gke https://github.com/salrashid123/gcegrpc/

hjfreyer commented 3 years ago

@stanley-cheung That config doesn't run on v1.17.

I did manage to get it working, but it took a lot of guesswork. Here's the config I landed on: https://github.com/hjfreyer/pictophone-be/blob/d72c869535f25ff289e806e9edac26d08f13eeba/config/envoy/envoy.yaml

robcecil commented 3 years ago

@hjfreyer The yaml doesn't seem complete... How does it work? I don't see any tls_certificates, nor trusted_ca... Shouldn't you have something like this under transport_sockets:

    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        # sni: server-f2dxgbokta-uc.a.run.app
        common_tls_context:
          tls_certificates:
            - certificate_chain: { filename: "/etc/server_crt.pem" }
              private_key: { filename: "/etc/server_key.pem" }
          validation_context:
            match_subject_alt_names:
            - exact: "server-f2dxgbokta-uc.a.run.app"
            trusted_ca:
              filename: /etc/ssl/certs/ca-certificates/my_ca.crt

ghost commented 3 years ago

I also hope we can have an official tutorial to setup grpc-web -> envoy proxy -> backend

jereloh commented 2 years ago

@stanley-cheung That config doesn't run on v1.17.

I did manage to get it working, but it took a lot of guesswork. Here's the config I landed on: https://github.com/hjfreyer/pictophone-be/blob/d72c869535f25ff289e806e9edac26d08f13eeba/config/envoy/envoy.yaml

Thank you for sharing this! I manage to also implement based on your config. 👍

santiihoyos commented 2 years ago

Hi, I'm configuring envoy with gRPC microservice over TLS connection on Flutter apps(including web) + dart Service. Do you know some reference to help me with envoy TLS setup for web app?. TLS on microservice to "native apps" (android, ios, mac, windows, linux are working).

You can find test project on: https://github.com/santiihoyos/gRPC_Chat_flutter branch: TLS_Support envoy setup at: server/setyup/envoy

I promise create medium blog to explain how does it works?

Thanks

joojscript commented 2 years ago

Hey @santiihoyos!

I'm trying to as well, unfortunately, I think I came close, but still can't use on web, did not test on "native" yet, but I'm trying on flutter as well.

My .yaml file is as follows:

admin:
  address:
    socket_address: {address: 0.0.0.0, port_value: 9901}

static_resources:
  listeners:
  - name: listener1
    address:
      socket_address: {address: 0.0.0.0, port_value: 8080}
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: grpc_json
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              # NOTE: by default, matching happens based on the gRPC route, and not on the incoming request path.
              # Reference: https://envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_json_transcoder_filter#route-configs-for-transcoded-requests
              - match: {prefix: "/auth.SignService"}
                route: {cluster: grpc, timeout: 60s}
          http_filters:
          - name: envoy.filters.http.grpc_json_transcoder
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder
              proto_descriptor: "/etc/envoy/proto.pb"
              services: ["auth.SignService"]
              print_options:
                add_whitespace: true
                always_print_primitive_fields: true
                always_print_enums_as_ints: false
                preserve_proto_field_names: false
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificates:
            - certificate_chain: 
                filename: "/etc/envoy/certs/cert.crt"
              private_key: 
                filename: "/etc/envoy/certs/cert.key"

  clusters:
  - name: grpc
    type: LOGICAL_DNS
    lb_policy: ROUND_ROBIN
    dns_lookup_family: V4_ONLY
    typed_extension_protocol_options:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        explicit_http_config:
          http2_protocol_options: {}
    load_assignment:
      cluster_name: grpc
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 192.168.0.123
                port_value: 5000
    # transport_socket:
    #   name: envoy.transport_sockets.tls
    #   typed_config:
    #     "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
    #     common_tls_context:
    #       # tls_certificates:
    #       # - certificate_chain: {"filename": "/etc/envoy/certs/cert.crt"}
    #       #   private_key: {"filename": "/etc/envoy/certs/cert.key"}
    #       #   ocsp_staple: {"filename": "/etc/envoy/certs/ca.key"}
    #       validation_context:
    #         # match_typed_subject_alt_names:
    #         # - san_type: DNS
    #         #   matcher:
    #         #     exact: "foo"
    #         trusted_ca:
    #           filename: "/etc/envoy/certs/ca.crt"
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        common_tls_context:
          tls_certificates:
          # The following self-signed certificate pair is generated using:
          # $ openssl req -x509 -newkey rsa:2048 -keyout a/front-proxy-key.pem -out  a/front-proxy-crt.pem -days 3650 -nodes -subj '/CN=front-envoy'
          #
          # Instead of feeding it as an inline_string, certificate pair can also be fed to Envoy
          # via filename. Reference: https://envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/base.proto#config-core-v3-datasource.
          #
          # Or in a dynamic configuration scenario, certificate pair can be fetched remotely via
          # Secret Discovery Service (SDS). Reference: https://envoyproxy.io/docs/envoy/latest/configuration/security/secret.
          - certificate_chain: 
              filename: "/etc/envoy/certs/cert.crt"
            private_key: 
              filename: "/etc/envoy/certs/cert.key"
          validation_context:
            match_subject_alt_names:
            - exact: "*"
            trusted_ca:
              filename: "/etc/envoy/certs/ca.crt"

cbrunnkvist commented 3 weeks ago

How can this issue still be open three years down the line?

grpc / grpc-web

Example of envoy with HTTPS connection to grpc backend #1022