Envoy 1.13.0: Didn't find a registered implementation for name: 'envoy.ext_authz'

[mabukhovsky]

Title: Envoy 1.13.0: Didn't find a registered implementation for name: 'envoy.ext_authz'

Description: I'm following envoy Envoy 1.13.0 documentation to build dynamic forward proxy that is using ExtAuthz cluster to verify requests (see relevant links). It is saying that: This filter should be configured with the name envoy.ext_authz However I'm hitting: Didn't find a registered implementation for name: 'envoy.ext_authz' error and Envoy exits with code 1.

If I remove filter block from my cluster, it starts and works fine.

[optional Relevant Links:] https://www.envoyproxy.io/docs/envoy/v1.13.0/configuration/http/http_filters/ext_authz_filter

Config:

admin:
  access_log_path: %WD_HOME%/logs/admin_access.log
  address:
    socket_address: { address: 127.0.0.1, port_value: %ADMIN_PORT% }

static_resources:
  listeners:
    - name: exposed_admin_listener
      address:
        socket_address: { address: 0.0.0.0, port_value: %EXPOSED_ADMIN_PORT% }
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: AUTO
                access_log:
                  - name: envoy.file_access_log
                    typed_config:
                      "@type": type.googleapis.com/envoy.config.accesslog.v2.FileAccessLog
                      path: %WD_HOME%/logs/exposed_admin_access.log
                route_config:
                  name: envoy_admin
                  virtual_hosts:
                    - name: envoy_admin
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/app_info/metrics" }
                          route: { cluster: exposed_admin, prefix_rewrite: "/stats/prometheus" }
                http_filters:
    - name: envoy.router
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 10324
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: dynamic_forward_proxy_cluster
                          per_filter_config:
                            envoy.filters.http.dynamic_forward_proxy:
                              auto_host_rewrite_header: "X-Host-Port"
                        - match:
                            prefix: "/"
                          route:
                            cluster: dynamic_forward_proxy_cluster
                http_filters:
                  - name: envoy.filters.http.dynamic_forward_proxy
                    typed_config:
                      "@type": type.googleapis.com/envoy.config.filter.http.dynamic_forward_proxy.v2alpha.FilterConfig
                      dns_cache_config:
                        name: dynamic_forward_proxy_cache_config
                        dns_lookup_family: V4_ONLY
                  - name: envoy.router

  clusters:
    - name: xds_cluster
      connect_timeout: 5s
      type: LOGICAL_DNS
      dns_lookup_family: V4_ONLY
      lb_policy: ROUND_ROBIN
      http2_protocol_options: {}
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.api.v2.auth.UpstreamTlsContext
          common_tls_context:
            validation_context:
              trusted_ca: {filename: /etc/ssl/certs/ca-bundle.crt}
      dns_refresh_rate:
        seconds: 3600
      load_assignment:
        cluster_name: xds_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: %XDS_HOST%
                      port_value: %XDS_PORT%
    - name: dynamic_forward_proxy_cluster
      connect_timeout: 5s
      lb_policy: CLUSTER_PROVIDED
      cluster_type:
        name: envoy.clusters.dynamic_forward_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.config.cluster.dynamic_forward_proxy.v2alpha.ClusterConfig
          dns_cache_config:
            name: dynamic_forward_proxy_cache_config
            dns_lookup_family: V4_ONLY
      filters:
        - name: envoy.ext_authz
          typed_config:
            "@type": "type.googleapis.com/envoy.config.filter.http.ext_authz.v2.ExtAuthz"
            grpc_service:
              envoy_grpc:
                cluster_name: xds_cluster
              timeout: 5s

    - name: exposed_admin
      connect_timeout: 0.250s
      type: STATIC
      hosts:
        - socket_address: { address: 127.0.0.1, port_value: %ADMIN_PORT% }

*Logs*:
proxy_1        | /custom-envoy/envoy -c /usr/local/workday-envoy/envoy.yaml --component-log-level main:info,http:trace,http2:trace,config:trace,filter:trace,router:trace,upstream:trace,client:trace,connection:trace,grpc:trace --config-yaml {'admin':{'address':{'socket_address':{'address':'0.0.0.0'}}}}
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:251] initializing epoch 0 (hot restart version=11.104)
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:253] statically linked extensions:
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:255]   envoy.grpc_credentials: envoy.grpc_credentials.aws_iam, envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:255]   envoy.filters.network: envoy.client_ssl_auth, envoy.echo, envoy.ext_authz, envoy.filters.network.dubbo_proxy, envoy.filters.network.kafka_broker, envoy.filters.network.local_ratelimit, envoy.filters.network.mysql_proxy, envoy.filters.network.rbac, envoy.filters.network.sni_cluster, envoy.filters.network.thrift_proxy, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:255]   envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.tap, envoy.transport_sockets.tls, raw_buffer, tls
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:255]   envoy.thrift_proxy.filters: envoy.filters.thrift.rate_limit, envoy.filters.thrift.router
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.filters: envoy.filters.dubbo.router
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:255]   envoy.filters.listener: envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector
proxy_1        | [2020-03-25 21:34:32.652][1][info][main] [source/server/server.cc:255]   envoy.tracers: envoy.dynamic.ot, envoy.lightstep, envoy.tracers.datadog, envoy.tracers.opencensus, envoy.tracers.xray, envoy.zipkin
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.tap, envoy.transport_sockets.tls, raw_buffer, tls
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.health_checkers: envoy.health_checkers.redis
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.thrift_proxy.transports: auto, framed, header, unframed
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.route_matchers: default
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.clusters: envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.dynamic_forward_proxy, envoy.clusters.redis
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.previous_hosts
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.protocols: dubbo
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.retry_priorities: envoy.retry_priorities.previous_priorities
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.serializers: dubbo.hessian2
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.resolvers: envoy.ip
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.access_loggers: envoy.file_access_log, envoy.http_grpc_access_log, envoy.tcp_grpc_access_log
proxy_1        | [2020-03-25 21:34:32.653][1][info][main] [source/server/server.cc:255]   envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter
proxy_1        | [2020-03-25 21:34:32.654][1][info][main] [source/server/server.cc:255]   envoy.filters.http: envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.fault, envoy.filters.http.adaptive_concurrency, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_stats, envoy.filters.http.header_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.rbac, envoy.filters.http.tap, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.gzip, envoy.health_check, envoy.http_dynamo_filter, envoy.ip_tagging, envoy.lua, envoy.rate_limit, envoy.router, envoy.squash
proxy_1        | [2020-03-25 21:34:32.654][1][info][main] [source/server/server.cc:255]   envoy.filters.udp_listener: envoy.filters.udp_listener.udp_proxy
proxy_1        | [2020-03-25 21:34:32.654][1][info][main] [source/server/server.cc:255]   envoy.resource_monitors: envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource
proxy_1        | [2020-03-25 21:34:32.654][1][info][main] [source/server/server.cc:255]   envoy.stats_sinks: envoy.dog_statsd, envoy.metrics_service, envoy.stat_sinks.hystrix, envoy.statsd
proxy_1        | [2020-03-25 21:34:32.654][1][info][main] [source/server/server.cc:255]   envoy.udp_listeners: raw_udp_listener
proxy_1        | [2020-03-25 21:34:32.664][1][warning][misc] [source/common/protobuf/utility.cc:441] Using deprecated option 'envoy.api.v2.Cluster.hosts' from file cluster.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details.
proxy_1        | [2020-03-25 21:34:32.664][1][warning][misc] [source/common/protobuf/utility.cc:441] Using deprecated option 'envoy.config.trace.v2.Tracing.Http.config' from file trace.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details.
proxy_1        | [2020-03-25 21:34:32.667][1][info][main] [source/server/server.cc:336] admin address: 0.0.0.0:10327
proxy_1        | [2020-03-25 21:34:32.669][1][info][main] [source/server/server.cc:455] runtime: layers:
proxy_1        |   - name: base
proxy_1        |     static_layer:
proxy_1        |       {}
proxy_1        |   - name: admin
proxy_1        |     admin_layer:
proxy_1        |       {}
proxy_1        | [2020-03-25 21:34:32.670][1][info][config] [source/server/configuration_impl.cc:62] loading 0 static secret(s)
proxy_1        | [2020-03-25 21:34:32.670][1][info][config] [source/server/configuration_impl.cc:68] loading 3 cluster(s)
proxy_1        | [2020-03-25 21:34:32.671][63][debug][grpc] [source/common/grpc/google_async_client_impl.cc:45] completionThread running
proxy_1        | [2020-03-25 21:34:32.684][1][debug][upstream] [source/common/upstream/upstream_impl.cc:776]   upstream filter #0:
proxy_1        | [2020-03-25 21:34:32.684][1][debug][upstream] [source/common/upstream/upstream_impl.cc:777]     name: envoy.ext_authz
proxy_1        | [2020-03-25 21:34:32.687][1][critical][main] [source/server/server.cc:94] error initializing configuration '/usr/local/workday-envoy/envoy.yaml': Didn't find a registered implementation for name: 'envoy.ext_authz'
proxy_1        | [2020-03-25 21:34:32.687][1][debug][grpc] [source/common/grpc/google_async_client_impl.cc:35] Joining completionThread
proxy_1        | [2020-03-25 21:34:32.687][63][debug][grpc] [source/common/grpc/google_async_client_impl.cc:68] completionThread exiting
proxy_1        | [2020-03-25 21:34:32.688][1][debug][grpc] [source/common/grpc/google_async_client_impl.cc:37] Joined completionThread
proxy_1        | [2020-03-25 21:34:32.689][1][info][main] [source/server/server.cc:595] exiting
proxy_1        | Didn't find a registered implementation for name: 'envoy.ext_authz'

Note: If there are privacy concerns, sanitize the data prior to sharing.

Call Stack:

If the Envoy binary is crashing, a call stack is required. Please refer to the Bazel Stack trace documentation.

[mattklein123]

cc @zuercher I think you are using a newer build with the older docs, but I'm not sure.

[mabukhovsky]

If I start Envoy without a filter I can clearly see that's I'm using 1.13.0:

{
  "configs": [
    {
      "@type": "type.googleapis.com/envoy.admin.v3.BootstrapConfigDump",
      "bootstrap": {
        "node": {
          "id": "9dc0633771a0",
          "cluster": "s-envoy",
          "metadata": {
            "host": "9dc0633771a0",
            "instance": "9dc0633771a0s-envoy",
            "port": 10324,
            "admin_port": 10327
          },
          "hidden_envoy_deprecated_build_version": "bb7ceff4c3c5bd4555dff28b6e56d27f2f8be0a7/1.13.0/Clean/RELEASE/BoringSSL",
          "user_agent_name": "envoy",
          "user_agent_build_version": {
            "version": {
              "major_number": 1,
              "minor_number": 13
            },
            "metadata": {
              "ssl.version": "BoringSSL",
              "revision.status": "Clean",
              "revision.sha": "bb7ceff4c3c5bd4555dff28b6e56d27f2f8be0a7",
              "build.type": "RELEASE"
            }
          },

[dio]

    - name: dynamic_forward_proxy_cluster
      ...
      filters:
      - name: envoy.ext_authz
      ...

Hum, seems like that filters: field only accepts a list of NamedUpstreamNetworkFilters? cc. @htuch

[mabukhovsky]

@zuercher @dio @htuch Thanks for looking into my issue, guys. Truly appreciate your help as this is a blocker for a team. We are trying to build dynamic forward proxy with ext_authz requests authorization. Please let us know if this is possible, or there are constraints that do not allow doing it.

[zuercher]

To clarify some more: There is no support for http filters on upstream clusters. As @dio mentioned only upstream network filters are supported and there aren't actually any shipped with Envoy (there's one defined for tests only). Issue #10455 tracks adding support for upstream http filters, at which point use the ext_authz filter becomes theoretically possible (depending on what limitations, if any, are placed on upstream http filters).

[dio]

One example of network filters that can be attached to that is istio.metadata_exchange wasm filter. https://github.com/istio/proxy/blob/1d997c17e2321d43565c80eb828f7bdcb78de7e2/src/envoy/tcp/metadata_exchange/config.h#L49-L71

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[rebrendov]

Hi,

I'm having a similar issue. Except i try to configure RBAC filter envoy.filters.network.rbac for a cluster and get Didn't find a registered implementation for name: 'envoy.filters.network.rbac'.

"version": "bb7ceff4c3c5bd4555dff28b6e56d27f2f8be0a7/1.13.0/Clean/RELEASE/BoringSSL",

As @dio mentioned only upstream network filters are supported and there aren't actually any shipped with Envoy (there's one defined for tests only)

@zuercher please, correct me if i'm wrong - does this mean that there needs to be a separate upstream filter with similar functionality to network.rbac but exclusively for upstream?

Edit: based on #11015 the answer seems to be - yes.

[zuercher]

It looks UpstreamNetworkFilters implement the same interface as a regular network filter, but have a separate registration. Someone would have to evaluate whether the RBAC filter operates correctly in the upstream context and, if so, add a registration for it.

@kyessenov could you confirm that?

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.