Enable http_filters.envoy.filters.http.grpc_http1_reverse_bridge only for one service

[sp-manuel-jurado]

Title: Enable http_filters.envoy.filters.http.grpc_http1_reverse_bridge only for one service

Description:

Would be possible to enable envoy.filters.http.grpc_http1_reverse_bridge for service1 only? Probably I missed some config?

• Service1 is in http1 and requires the reverse bridge
• Service2 is gRPC and does not require the reverse bridge (but is blocked by the .grpc_http1_reverse_bridge)

Note:

I would like to use the same port for both services.

Config:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      protocol: TCP
      address: 127.0.0.1
      port_value: 9901
static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address:
          protocol: TCP
          address: 0.0.0.0
          port_value: 10000
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: AUTO
                route_config:
                  name: backend
                  virtual_hosts:
                    - name: backend
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/service2/" }
                          route: { host_rewrite: grpc, cluster: grpc, timeout: 59.99s }
                        - match: { prefix: "/service1/" }
                          route: { host_rewrite: nginx, cluster: backend, timeout: 59.99s }
                http_filters:
                  - name: envoy.filters.http.grpc_http1_reverse_bridge
                    config:
                      content_type: application/grpc+proto
                      withhold_grpc_frames: true
                  - name: envoy.router
                    typed_config: {}
  clusters:
  - name: backend
    connect_timeout: 59.99s
    type: logical_dns
    dns_lookup_family: v4_only
    lb_policy: round_robin
    load_assignment:
      cluster_name: backend
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: nginx
                    port_value: 80
  - name: grpc
    connect_timeout: 0.25s
    type: strict_dns
    lb_policy: round_robin
    http2_protocol_options: {}
    load_assignment:
      cluster_name: grpc
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: grpc
                    port_value: 10005

[sp-manuel-jurado]

I'm using v1.11.1 envoy version

[snowp]

Not currently, but it wouldn't be too hard to make it possible to enable this on a per route basis

[sp-manuel-jurado]

Thx @snowp I would be very interested in this improvement. Anything can I do?

[snowp]

The change wouldn't be too complicated: a) have the filter read the filter config doing something like this https://github.com/envoyproxy/envoy/blob/master/source/extensions/filters/http/fault/fault_filter.cc#L104 b) use that to determine whether the filter is enabled here https://github.com/envoyproxy/envoy/blob/master/source/extensions/filters/http/grpc_http1_reverse_bridge/filter.cc#L63. We might want a enabled flag on the proto so that we can just disable it directly (see my TODO in there) c) add an integration test to https://github.com/envoyproxy/envoy/blob/master/test/extensions/filters/http/grpc_http1_reverse_bridge/reverse_bridge_integration_test.cc that verifies that we're picking up the route config

with this you would be able to add a route matcher that uses the per_filter_config to disable the filter for a specific route:

per_filter_config:
  envoy.filters.http.grpc_http1_reverse_bridge: 
    enabled: false

[sp-manuel-jurado]

Thanks for the tips @snowp (and sorry for the delay).

I'll try to do the PR ASAP but before that, I have some doubts:

I guess doing something like (in filter.cc):

...
#include "envoy/config/filter/http/grpc_http1_reverse_bridge/v2alpha1/config.pb.validate.h"
...
Http::FilterHeadersStatus Filter::decodeHeaders(Http::HeaderMap& headers, bool end_stream) {
  // Short circuit if header only.
  if (end_stream) {
    return Http::FilterHeadersStatus::Continue;
  }

  // Check enable by route
  if (decoder_callbacks_->route() && decoder_callbacks_->route()->routeEntry()) {
    const std::string& name = Extensions::HttpFilters::HttpFilterNames::get().GrpcHttp1ReverseBridge;
    const auto* route_entry = decoder_callbacks_->route()->routeEntry();
    const auto* config_per_route = route_entry->perFilterConfigTyped<envoy::config::filter::http::grpc_http1_reverse_bridge::v2alpha1::FilterConfig>(name);
    ENVOY_LOG_MISC(info, "grpc http1 reverse bridge config per route");
    if (config_per_route != nullptr) {
        ENVOY_LOG_MISC(info, "config_per_route != nullptr");
        if (!config_per_route->enabled()) {
            enabled_ = false;
            ENVOY_LOG_MISC(info, "disabled");
            return Http::FilterHeadersStatus::Continue;
        }
    }
  }
...

and in config.proto add enabled field (envoy.config.filter.http.grpc_http1_reverse_bridge.v2alpha1) (if not config.yaml validation fails in execution time):

...
bool enabled = 3;
...

I'm checking it using a local build with this config.yaml but It seems not working... I'm doing something wrong?

admin:
  access_log_path: /dev/stdout
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 9901
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 80
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          access_log:
          - name: envoy.file_access_log
            config:
              path: /dev/stdout
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/service1/"
                route:
                  host_rewrite: ec2-52-91-84-220.compute-1.amazonaws.com
                  cluster: service1
                  timeout: 50.00s
              - match:
                  prefix: "/service2/"
                route:
                  host_rewrite: service2
                  cluster: grpc
                  timeout: 50.00s
                per_filter_config:
                  envoy.filters.http.grpc_http1_reverse_bridge:
                    content_type: application/grpc+proto
                    withhold_grpc_frames: true
                    enabled: false
          http_filters:
          - name: envoy.filters.http.grpc_http1_reverse_bridge
            config:
              content_type: application/grpc+proto
              withhold_grpc_frames: true
              enabled: true
          - name: envoy.router
  clusters:
  - name: sg_backend_dynenvoy
    connect_timeout: 50.00s
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    hosts:
      - socket_address:
          address: service1
          port_value: 4630
  - name: grpc
    connect_timeout: 50.00s
    type: strict_dns
    lb_policy: round_robin
    http2_protocol_options: {}
    load_assignment:
      cluster_name: grpc
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: service2
                    port_value: 10005

Logs:

./bazel-bin/source/exe/envoy-static --config-path bazel-bin/source/exe/config.yaml
[2019-09-03 16:07:28.546][847372][info][main] [source/server/server.cc:242] initializing epoch 0 (hot restart version=disabled)
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:244] statically linked extensions:
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:246]   access_loggers: envoy.file_access_log,envoy.http_grpc_access_log,envoy.tcp_grpc_access_log
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:249]   filters.http: envoy.buffer,envoy.cors,envoy.csrf,envoy.ext_authz,envoy.fault,envoy.filters.http.dynamic_forward_proxy,envoy.filters.http.grpc_http1_reverse_bridge,envoy.filters.http.header_to_metadata,envoy.filters.http.jwt_authn,envoy.filters.http.original_src,envoy.filters.http.rbac,envoy.filters.http.tap,envoy.grpc_http1_bridge,envoy.grpc_json_transcoder,envoy.grpc_web,envoy.gzip,envoy.health_check,envoy.http_dynamo_filter,envoy.ip_tagging,envoy.lua,envoy.rate_limit,envoy.router,envoy.squash
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:252]   filters.listener: envoy.listener.http_inspector,envoy.listener.original_dst,envoy.listener.original_src,envoy.listener.proxy_protocol,envoy.listener.tls_inspector
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:255]   filters.network: envoy.client_ssl_auth,envoy.echo,envoy.ext_authz,envoy.filters.network.dubbo_proxy,envoy.filters.network.mysql_proxy,envoy.filters.network.rbac,envoy.filters.network.sni_cluster,envoy.filters.network.thrift_proxy,envoy.filters.network.zookeeper_proxy,envoy.http_connection_manager,envoy.mongo_proxy,envoy.ratelimit,envoy.redis_proxy,envoy.tcp_proxy
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:257]   stat_sinks: envoy.dog_statsd,envoy.metrics_service,envoy.stat_sinks.hystrix,envoy.statsd
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:259]   tracers: envoy.dynamic.ot,envoy.lightstep,envoy.tracers.datadog,envoy.tracers.opencensus,envoy.zipkin
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:262]   transport_sockets.downstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:265]   transport_sockets.upstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls
[2019-09-03 16:07:28.547][847372][info][main] [source/server/server.cc:271] buffer implementation: new
[2019-09-03 16:07:28.571][847372][info][main] [source/server/server.cc:331] admin address: 0.0.0.0:9901
[2019-09-03 16:07:28.578][847372][info][main] [source/server/server.cc:445] runtime: layers:
  - name: base
    static_layer:
      {}
  - name: admin
    admin_layer:
      {}
[2019-09-03 16:07:28.578][847372][info][config] [source/server/configuration_impl.cc:62] loading 0 static secret(s)
[2019-09-03 16:07:28.578][847372][info][config] [source/server/configuration_impl.cc:68] loading 2 cluster(s)
[2019-09-03 16:07:28.620][847372][info][config] [source/server/configuration_impl.cc:72] loading 1 listener(s)
[2019-09-03 16:07:28.637][847372][info][config] [source/server/configuration_impl.cc:97] loading tracing configuration
[2019-09-03 16:07:28.637][847372][info][config] [source/server/configuration_impl.cc:117] loading stats sink configuration
[2019-09-03 16:07:28.637][847372][info][main] [source/server/server.cc:530] starting main dispatch loop
[2019-09-03 16:07:28.867][847372][info][upstream] [source/common/upstream/cluster_manager_impl.cc:161] cm init: all clusters initialized
[2019-09-03 16:07:28.867][847372][info][main] [source/server/server.cc:513] all clusters initialized. initializing init manager
[2019-09-03 16:07:28.867][847372][info][config] [source/server/listener_manager_impl.cc:789] all dependencies initialized. starting workers

[2019-09-03 16:07:39.874][847380][info][misc] [source/extensions/filters/http/grpc_http1_reverse_bridge/filter.cc:64] grpc http1 reverse bridge config per route
[2019-09-03T14:07:39.868Z] "POST ...

[sp-manuel-jurado]

@snowp I made a WIP PR: https://github.com/envoyproxy/envoy/pull/8145 Before updating the tests related, Am I on the right way?

[sp-manuel-jurado]

@snowp I made this final PR: https://github.com/envoyproxy/envoy/pull/8645