envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.99k stars 4.81k forks source link

Passive healthchecking not respecting ejection time #31070

Closed leandrocurioso closed 11 months ago

leandrocurioso commented 11 months ago

Hi Envoy Team,

I'm doing active + passive health cheking in my envoy but I'm getting a strange behaviour detailed below:

Scenario

Effect

For some reason (maybe I missed that in the docs), if lb_endpointB returns 2 times http response code 500 envoy removes the endpoint from receiving request BUT doesn't respect the base_ejection_time in less then 10 or 15s it is added again to receive requests.

I don't know what is happening maybe the active health check is adding it again to active mode in backgoround? I can't see any logs about it (current log level equals to info).

When I remove the active health check the outlier respects the ejection time but for some reason if I have active health check enabled this strange behaviour happen.

What I Expected

I thought no matter what the passive health check would keep the endpoint out of receiving. What could possibly put the endpoint live again if the base_ejection_time is not reached?

Thanks for your time!

suniltheta commented 11 months ago

Same as https://github.com/envoyproxy/envoy/issues/31037

This was a behavior change introduced in v1.26. Which Envoy version are you using? If it is v1.26 or v1.27 then your can set envoy.reloadable_features_successful_active_health_check_uneject_host to false. Or if it is v1.28+ then see the Note in https://www.envoyproxy.io/docs/envoy/v1.28.0/intro/arch_overview/upstream/outlier.html#grpc

leandrocurioso commented 11 months ago

I am using 1.27.1 but this setting is not available. @suniltheta

As mentioned here: [https://www.envoyproxy.io/docs/envoy/v1.27.1/intro/arch_overview/upstream/outlier.html#grpc]()

Is it possible in that version to configure so the active health check do not mess with the outlier?

suniltheta commented 11 months ago

in v1.27.1 you would have to set the feature flag envoy.reloadable_features_successful_active_health_check_uneject_host to false.

https://github.com/envoyproxy/envoy/blob/v1.27.1/source/common/runtime/runtime_features.cc#L77

leandrocurioso commented 11 months ago

How do I add this config in the file? Should be at cluster level? @suniltheta

suniltheta commented 11 months ago

Do you include runtime (feature flags) in your configuration? https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime.html

One way to include is add it in static bootstrap file. Below is a sample

layeredRuntime:
  layers:
  - name: static_layer_0
    staticLayer:
      envoy.reloadable_features.successful_active_health_check_uneject_host: false
  - adminLayer: {}
    name: admin_layer
leandrocurioso commented 11 months ago

It worked! Thanks a lot for your help =]

leandrocurioso commented 11 months ago

I added this config and is respecting the base time but as the endpoint keeps receiving 500 the total ejection time is not being multiplied.

For instance I have bass time equal to 30s and max ejection to 300s.

If the endpoint is ejected the first time it should respect the base time so 30s. (This is working fine) but for the next ejection should be 30sx2=60s. But this is not being respected, the ejection time keep being 60s. Am I missing something? @suniltheta

suniltheta commented 11 months ago

May be this is a bug that has to be looked into. Can you confirm if this was working prior to v1.26.0? Dose the ejection time increase when Active health check is disabled?

Can you update the title and the description to capture the new issue that you identified?

leandrocurioso commented 11 months ago

I will make a test and return here.

leandrocurioso commented 11 months ago

@suniltheta

I enabled outlier logging and I believe this is not working properly for the versions below:

The property secs_since_last_action says everything!

1.26.0

2023-11-28 17:17:20 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.80.2:80](http://192.168.80.2/)","action":"EJECT","num_ejections":1,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:17:18.045Z"}
2023-11-28 17:18:20 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.80.2:80](http://192.168.80.2/)","action":"UNEJECT","num_ejections":1,"enforced":false,"timestamp":"2023-11-28T20:18:18.984Z","secs_since_last_action":"60"}
2023-11-28 17:18:30 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.80.2:80](http://192.168.80.2/)","action":"EJECT","num_ejections":2,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:18:22.781Z","secs_since_last_action":"3"}
2023-11-28 17:19:30 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.80.2:80](http://192.168.80.2/)","action":"UNEJECT","num_ejections":2,"enforced":false,"timestamp":"2023-11-28T20:19:23.022Z","secs_since_last_action":"60"}
2023-11-28 17:19:30 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.80.2:80](http://192.168.80.2/)","action":"EJECT","num_ejections":3,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:19:25.035Z","secs_since_last_action":"2"}
2023-11-28 17:20:30 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.80.2:80](http://192.168.80.2/)","action":"UNEJECT","num_ejections":3,"enforced":false,"timestamp":"2023-11-28T20:20:25.045Z","secs_since_last_action":"60"}
2023-11-28 17:20:30 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.80.2:80](http://192.168.80.2/)","action":"EJECT","num_ejections":4,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:20:29.033Z","secs_since_last_action":"3"}

1.25.11

2023-11-28 17:22:35 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"EJECT","num_ejections":1,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:22:34.146Z"}
2023-11-28 17:23:35 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"UNEJECT","num_ejections":1,"enforced":false,"timestamp":"2023-11-28T20:23:34.589Z","secs_since_last_action":"60"}
2023-11-28 17:23:45 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"EJECT","num_ejections":2,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:23:38.143Z","secs_since_last_action":"3"}
2023-11-28 17:24:45 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"UNEJECT","num_ejections":2,"enforced":false,"timestamp":"2023-11-28T20:24:38.622Z","secs_since_last_action":"60"}
2023-11-28 17:24:45 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"EJECT","num_ejections":3,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:24:42.138Z","secs_since_last_action":"3"}
2023-11-28 17:25:45 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"UNEJECT","num_ejections":3,"enforced":false,"timestamp":"2023-11-28T20:25:42.652Z","secs_since_last_action":"60"}
2023-11-28 17:25:55 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"EJECT","num_ejections":4,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:25:46.140Z","secs_since_last_action":"3"}
2023-11-28 17:26:55 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"UNEJECT","num_ejections":4,"enforced":false,"timestamp":"2023-11-28T20:26:46.690Z","secs_since_last_action":"60"}
2023-11-28 17:26:55 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.96.3:80](http://192.168.96.3/)","action":"EJECT","num_ejections":5,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:26:50.140Z","secs_since_last_action":"3"}

1.24.12

2023-11-28 17:29:58 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.112.5:80](http://192.168.112.5/)","action":"EJECT","num_ejections":1,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:29:55.520Z"}
2023-11-28 17:30:58 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.112.5:80](http://192.168.112.5/)","action":"UNEJECT","num_ejections":1,"enforced":false,"timestamp":"2023-11-28T20:30:56.272Z","secs_since_last_action":"60"}
2023-11-28 17:31:08 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.112.5:80](http://192.168.112.5/)","action":"EJECT","num_ejections":2,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:30:59.494Z","secs_since_last_action":"3"}
2023-11-28 17:32:08 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.112.5:80](http://192.168.112.5/)","action":"UNEJECT","num_ejections":2,"enforced":false,"timestamp":"2023-11-28T20:32:00.297Z","secs_since_last_action":"60"}
2023-11-28 17:32:08 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.112.5:80](http://192.168.112.5/)","action":"EJECT","num_ejections":3,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:32:03.860Z","secs_since_last_action":"3"}
2023-11-28 17:33:08 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.112.5:80](http://192.168.112.5/)","action":"UNEJECT","num_ejections":3,"enforced":false,"timestamp":"2023-11-28T20:33:04.329Z","secs_since_last_action":"60"}
2023-11-28 17:33:08 {"type":"CONSECUTIVE_5XX","cluster_name":"sample-shard","upstream_url":"[192.168.112.5:80](http://192.168.112.5/)","action":"EJECT","num_ejections":4,"enforced":true,"eject_consecutive_event":{},"timestamp":"2023-11-28T20:33:07.483Z","secs_since_last_action":"3"}
suniltheta commented 11 months ago

So this bug has nothing to do with having active health check uneject outlier detection ejected hosts. Checking if disabling active health check will also run into this issue?

I need to look into code myself to see where the bug is. I am not sure who is the expert in Envoy health check, who can quickly identify if this is a known issue or new one.

leandrocurioso commented 11 months ago

I will remove active health check and see if this still happens.

leandrocurioso commented 11 months ago

@suniltheta @cpakulski

Even with no active health check the behaviour persists.

I think can be something here: https://github.com/envoyproxy/envoy/blob/v1.27.1/source/common/upstream/outlier_detection_impl.cc#L354

cpakulski commented 11 months ago

@leandrocurioso Can you dump config to make sure that max ejection time is not capped to 60 secs?

leandrocurioso commented 11 months ago

@cpakulski thanks for getting envolved.

Config Dump:

{
 "configs": [
  {
   "@type": "type.googleapis.com/envoy.admin.v3.BootstrapConfigDump",
   "bootstrap": {
    "node": {
     "id": "1",
     "cluster": "cell-1",
     "user_agent_name": "envoy",
     "user_agent_build_version": {
      "version": {
       "major_number": 1,
       "minor_number": 27,
       "patch": 1
      },
      "metadata": {
       "ssl.version": "BoringSSL",
       "revision.status": "Clean",
       "build.type": "RELEASE",
       "revision.sha": "6b9db09c69965d5bfb37bdd29693f8b7f9e9e9ec"
      }
     },
     "extensions": [
      {
       "name": "envoy.filters.thrift.header_to_metadata",
       "category": "envoy.thrift_proxy.filters",
       "type_urls": [
        "envoy.extensions.filters.network.thrift_proxy.filters.header_to_metadata.v3.HeaderToMetadata"
       ]
      },
      {
       "name": "envoy.filters.thrift.payload_to_metadata",
       "category": "envoy.thrift_proxy.filters",
       "type_urls": [
        "envoy.extensions.filters.network.thrift_proxy.filters.payload_to_metadata.v3.PayloadToMetadata"
       ]
      },
      {
       "name": "envoy.filters.thrift.rate_limit",
       "category": "envoy.thrift_proxy.filters",
       "type_urls": [
        "envoy.extensions.filters.network.thrift_proxy.filters.ratelimit.v3.RateLimit"
       ]
      },
      {
       "name": "envoy.filters.thrift.router",
       "category": "envoy.thrift_proxy.filters",
       "type_urls": [
        "envoy.extensions.filters.network.thrift_proxy.router.v3.Router"
       ]
      },
      {
       "name": "envoy.filters.listener.http_inspector",
       "category": "envoy.filters.listener",
       "type_urls": [
        "envoy.extensions.filters.listener.http_inspector.v3.HttpInspector"
       ]
      },
      {
       "name": "envoy.filters.listener.local_ratelimit",
       "category": "envoy.filters.listener",
       "type_urls": [
        "envoy.extensions.filters.listener.local_ratelimit.v3.LocalRateLimit"
       ]
      },
      {
       "name": "envoy.filters.listener.original_dst",
       "category": "envoy.filters.listener",
       "type_urls": [
        "envoy.extensions.filters.listener.original_dst.v3.OriginalDst"
       ]
      },
      {
       "name": "envoy.filters.listener.original_src",
       "category": "envoy.filters.listener",
       "type_urls": [
        "envoy.extensions.filters.listener.original_src.v3.OriginalSrc"
       ]
      },
      {
       "name": "envoy.filters.listener.proxy_protocol",
       "category": "envoy.filters.listener",
       "type_urls": [
        "envoy.extensions.filters.listener.proxy_protocol.v3.ProxyProtocol"
       ]
      },
      {
       "name": "envoy.filters.listener.tls_inspector",
       "category": "envoy.filters.listener",
       "type_urls": [
        "envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector"
       ]
      },
      {
       "name": "envoy.listener.http_inspector",
       "category": "envoy.filters.listener"
      },
      {
       "name": "envoy.listener.original_dst",
       "category": "envoy.filters.listener"
      },
      {
       "name": "envoy.listener.original_src",
       "category": "envoy.filters.listener"
      },
      {
       "name": "envoy.listener.proxy_protocol",
       "category": "envoy.filters.listener"
      },
      {
       "name": "envoy.listener.tls_inspector",
       "category": "envoy.filters.listener"
      },
      {
       "name": "envoy.matching.common_inputs.environment_variable",
       "category": "envoy.matching.common_inputs",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.environment_variable.v3.Config"
       ]
      },
      {
       "name": "envoy.path.rewrite.uri_template.uri_template_rewriter",
       "category": "envoy.path.rewrite",
       "type_urls": [
        "envoy.extensions.path.rewrite.uri_template.v3.UriTemplateRewriteConfig"
       ]
      },
      {
       "name": "envoy.extensions.upstreams.http.v3.HttpProtocolOptions",
       "category": "envoy.upstream_options",
       "type_urls": [
        "envoy.extensions.upstreams.http.v3.HttpProtocolOptions"
       ]
      },
      {
       "name": "envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions",
       "category": "envoy.upstream_options",
       "type_urls": [
        "envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions"
       ]
      },
      {
       "name": "envoy.upstreams.http.http_protocol_options",
       "category": "envoy.upstream_options"
      },
      {
       "name": "envoy.upstreams.tcp.tcp_protocol_options",
       "category": "envoy.upstream_options"
      },
      {
       "name": "envoy.dog_statsd",
       "category": "envoy.stats_sinks"
      },
      {
       "name": "envoy.graphite_statsd",
       "category": "envoy.stats_sinks"
      },
      {
       "name": "envoy.metrics_service",
       "category": "envoy.stats_sinks"
      },
      {
       "name": "envoy.open_telemetry_stat_sink",
       "category": "envoy.stats_sinks"
      },
      {
       "name": "envoy.stat_sinks.dog_statsd",
       "category": "envoy.stats_sinks",
       "type_urls": [
        "envoy.config.metrics.v3.DogStatsdSink"
       ]
      },
      {
       "name": "envoy.stat_sinks.graphite_statsd",
       "category": "envoy.stats_sinks",
       "type_urls": [
        "envoy.extensions.stat_sinks.graphite_statsd.v3.GraphiteStatsdSink"
       ]
      },
      {
       "name": "envoy.stat_sinks.hystrix",
       "category": "envoy.stats_sinks",
       "type_urls": [
        "envoy.config.metrics.v3.HystrixSink"
       ]
      },
      {
       "name": "envoy.stat_sinks.metrics_service",
       "category": "envoy.stats_sinks",
       "type_urls": [
        "envoy.config.metrics.v3.MetricsServiceConfig"
       ]
      },
      {
       "name": "envoy.stat_sinks.open_telemetry",
       "category": "envoy.stats_sinks",
       "type_urls": [
        "envoy.extensions.stat_sinks.open_telemetry.v3.SinkConfig"
       ]
      },
      {
       "name": "envoy.stat_sinks.statsd",
       "category": "envoy.stats_sinks",
       "type_urls": [
        "envoy.config.metrics.v3.StatsdSink"
       ]
      },
      {
       "name": "envoy.stat_sinks.wasm",
       "category": "envoy.stats_sinks",
       "type_urls": [
        "envoy.extensions.stat_sinks.wasm.v3.Wasm"
       ]
      },
      {
       "name": "envoy.statsd",
       "category": "envoy.stats_sinks"
      },
      {
       "name": "envoy.transport_sockets.alts",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.alts.v3.Alts"
       ]
      },
      {
       "name": "envoy.transport_sockets.http_11_proxy",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.http_11_proxy.v3.Http11ProxyUpstreamTransport"
       ]
      },
      {
       "name": "envoy.transport_sockets.internal_upstream",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.internal_upstream.v3.InternalUpstreamTransport"
       ]
      },
      {
       "name": "envoy.transport_sockets.quic",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.quic.v3.QuicUpstreamTransport"
       ]
      },
      {
       "name": "envoy.transport_sockets.raw_buffer",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.raw_buffer.v3.RawBuffer"
       ]
      },
      {
       "name": "envoy.transport_sockets.starttls",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.starttls.v3.UpstreamStartTlsConfig"
       ]
      },
      {
       "name": "envoy.transport_sockets.tap",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.tap.v3.Tap"
       ]
      },
      {
       "name": "envoy.transport_sockets.tcp_stats",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.tcp_stats.v3.Config"
       ]
      },
      {
       "name": "envoy.transport_sockets.tls",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext"
       ]
      },
      {
       "name": "envoy.transport_sockets.upstream_proxy_protocol",
       "category": "envoy.transport_sockets.upstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.proxy_protocol.v3.ProxyProtocolUpstreamTransport"
       ]
      },
      {
       "name": "raw_buffer",
       "category": "envoy.transport_sockets.upstream"
      },
      {
       "name": "starttls",
       "category": "envoy.transport_sockets.upstream"
      },
      {
       "name": "tls",
       "category": "envoy.transport_sockets.upstream"
      },
      {
       "name": "envoy.load_balancing_policies.cluster_provided",
       "category": "envoy.load_balancing_policies",
       "type_urls": [
        "envoy.extensions.load_balancing_policies.cluster_provided.v3.ClusterProvided"
       ]
      },
      {
       "name": "envoy.load_balancing_policies.least_request",
       "category": "envoy.load_balancing_policies",
       "type_urls": [
        "envoy.extensions.load_balancing_policies.least_request.v3.LeastRequest"
       ]
      },
      {
       "name": "envoy.load_balancing_policies.maglev",
       "category": "envoy.load_balancing_policies",
       "type_urls": [
        "envoy.extensions.load_balancing_policies.maglev.v3.Maglev"
       ]
      },
      {
       "name": "envoy.load_balancing_policies.random",
       "category": "envoy.load_balancing_policies",
       "type_urls": [
        "envoy.extensions.load_balancing_policies.random.v3.Random"
       ]
      },
      {
       "name": "envoy.load_balancing_policies.ring_hash",
       "category": "envoy.load_balancing_policies",
       "type_urls": [
        "envoy.extensions.load_balancing_policies.ring_hash.v3.RingHash"
       ]
      },
      {
       "name": "envoy.load_balancing_policies.round_robin",
       "category": "envoy.load_balancing_policies",
       "type_urls": [
        "envoy.extensions.load_balancing_policies.round_robin.v3.RoundRobin"
       ]
      },
      {
       "name": "envoy.load_balancing_policies.subset",
       "category": "envoy.load_balancing_policies",
       "type_urls": [
        "envoy.extensions.load_balancing_policies.subset.v3.Subset"
       ]
      },
      {
       "name": "envoy.http.header_validators.envoy_default",
       "category": "envoy.http.header_validators",
       "type_urls": [
        "envoy.extensions.http.header_validators.envoy_default.v3.HeaderValidatorConfig"
       ]
      },
      {
       "name": "envoy.filters.connection_pools.tcp.generic",
       "category": "envoy.upstreams",
       "type_urls": [
        "envoy.extensions.upstreams.tcp.generic.v3.GenericConnectionPoolProto"
       ]
      },
      {
       "name": "dubbo.hessian2",
       "category": "envoy.dubbo_proxy.serializers"
      },
      {
       "name": "envoy.health_checkers.grpc",
       "category": "envoy.health_checkers",
       "type_urls": [
        "envoy.config.core.v3.HealthCheck.GrpcHealthCheck"
       ]
      },
      {
       "name": "envoy.health_checkers.http",
       "category": "envoy.health_checkers",
       "type_urls": [
        "envoy.config.core.v3.HealthCheck.HttpHealthCheck"
       ]
      },
      {
       "name": "envoy.health_checkers.redis",
       "category": "envoy.health_checkers",
       "type_urls": [
        "envoy.extensions.health_checkers.redis.v3.Redis"
       ]
      },
      {
       "name": "envoy.health_checkers.tcp",
       "category": "envoy.health_checkers",
       "type_urls": [
        "envoy.config.core.v3.HealthCheck.TcpHealthCheck"
       ]
      },
      {
       "name": "envoy.health_checkers.thrift",
       "category": "envoy.health_checkers",
       "type_urls": [
        "envoy.extensions.health_checkers.thrift.v3.Thrift"
       ]
      },
      {
       "name": "envoy.matching.inputs.cel_data_input",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "xds.type.matcher.v3.HttpAttributesCelMatchInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.destination_ip",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.DestinationIPInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.destination_port",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.DestinationPortInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.direct_source_ip",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.DirectSourceIPInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.dns_san",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.ssl.v3.DnsSanInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.request_headers",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.type.matcher.v3.HttpRequestHeaderMatchInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.request_trailers",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.type.matcher.v3.HttpRequestTrailerMatchInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.response_headers",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.type.matcher.v3.HttpResponseHeaderMatchInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.response_trailers",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.type.matcher.v3.HttpResponseTrailerMatchInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.server_name",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.ServerNameInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.source_ip",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.SourceIPInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.source_port",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.SourcePortInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.source_type",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.SourceTypeInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.status_code_class_input",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.type.matcher.v3.HttpResponseStatusCodeClassMatchInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.status_code_input",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.type.matcher.v3.HttpResponseStatusCodeMatchInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.subject",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.ssl.v3.SubjectInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.uri_san",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.ssl.v3.UriSanInput"
       ]
      },
      {
       "name": "query_params",
       "category": "envoy.matching.http.input",
       "type_urls": [
        "envoy.type.matcher.v3.HttpRequestQueryParamMatchInput"
       ]
      },
      {
       "name": "dubbo",
       "category": "envoy.dubbo_proxy.protocols"
      },
      {
       "name": "envoy.ip",
       "category": "envoy.resolvers"
      },
      {
       "name": "envoy.dynamic.ot",
       "category": "envoy.tracers"
      },
      {
       "name": "envoy.tracers.datadog",
       "category": "envoy.tracers",
       "type_urls": [
        "envoy.config.trace.v3.DatadogConfig"
       ]
      },
      {
       "name": "envoy.tracers.dynamic_ot",
       "category": "envoy.tracers",
       "type_urls": [
        "envoy.config.trace.v3.DynamicOtConfig"
       ]
      },
      {
       "name": "envoy.tracers.opencensus",
       "category": "envoy.tracers",
       "type_urls": [
        "envoy.config.trace.v3.OpenCensusConfig"
       ]
      },
      {
       "name": "envoy.tracers.opentelemetry",
       "category": "envoy.tracers",
       "type_urls": [
        "envoy.config.trace.v3.OpenTelemetryConfig"
       ]
      },
      {
       "name": "envoy.tracers.skywalking",
       "category": "envoy.tracers",
       "type_urls": [
        "envoy.config.trace.v3.SkyWalkingConfig"
       ]
      },
      {
       "name": "envoy.tracers.xray",
       "category": "envoy.tracers",
       "type_urls": [
        "envoy.config.trace.v3.XRayConfig"
       ]
      },
      {
       "name": "envoy.tracers.zipkin",
       "category": "envoy.tracers",
       "type_urls": [
        "envoy.config.trace.v3.ZipkinConfig"
       ]
      },
      {
       "name": "envoy.zipkin",
       "category": "envoy.tracers"
      },
      {
       "name": "envoy.matching.actions.format_string",
       "category": "envoy.matching.action",
       "type_urls": [
        "envoy.config.core.v3.SubstitutionFormatString"
       ]
      },
      {
       "name": "filter-chain-name",
       "category": "envoy.matching.action",
       "type_urls": [
        "google.protobuf.StringValue"
       ]
      },
      {
       "name": "envoy.config.validators.minimum_clusters",
       "category": "envoy.config.validators"
      },
      {
       "name": "envoy.config.validators.minimum_clusters_validator",
       "category": "envoy.config.validators",
       "type_urls": [
        "envoy.extensions.config.validators.minimum_clusters.v3.MinimumClustersValidator"
       ]
      },
      {
       "name": "envoy.bootstrap.internal_listener",
       "category": "envoy.bootstrap",
       "type_urls": [
        "envoy.extensions.bootstrap.internal_listener.v3.InternalListener"
       ]
      },
      {
       "name": "envoy.bootstrap.wasm",
       "category": "envoy.bootstrap",
       "type_urls": [
        "envoy.extensions.wasm.v3.WasmService"
       ]
      },
      {
       "name": "envoy.extensions.network.socket_interface.default_socket_interface",
       "category": "envoy.bootstrap",
       "type_urls": [
        "envoy.extensions.network.socket_interface.v3.DefaultSocketInterface"
       ]
      },
      {
       "name": "envoy.matching.custom_matchers.trie_matcher",
       "category": "envoy.matching.network.custom_matchers",
       "type_urls": [
        "xds.type.matcher.v3.IPMatcher"
       ]
      },
      {
       "name": "envoy.formatter.cel",
       "category": "envoy.formatter",
       "type_urls": [
        "envoy.extensions.formatter.cel.v3.Cel"
       ]
      },
      {
       "name": "envoy.formatter.metadata",
       "category": "envoy.formatter",
       "type_urls": [
        "envoy.extensions.formatter.metadata.v3.Metadata"
       ]
      },
      {
       "name": "envoy.formatter.req_without_query",
       "category": "envoy.formatter",
       "type_urls": [
        "envoy.extensions.formatter.req_without_query.v3.ReqWithoutQuery"
       ]
      },
      {
       "name": "envoy.network.dns_resolver.cares",
       "category": "envoy.network.dns_resolver",
       "type_urls": [
        "envoy.extensions.network.dns_resolver.cares.v3.CaresDnsResolverConfig"
       ]
      },
      {
       "name": "envoy.network.dns_resolver.getaddrinfo",
       "category": "envoy.network.dns_resolver",
       "type_urls": [
        "envoy.extensions.network.dns_resolver.getaddrinfo.v3.GetAddrInfoDnsResolverConfig"
       ]
      },
      {
       "name": "envoy.rate_limit_descriptors.expr",
       "category": "envoy.rate_limit_descriptors",
       "type_urls": [
        "envoy.extensions.rate_limit_descriptors.expr.v3.Descriptor"
       ]
      },
      {
       "name": "auto",
       "category": "envoy.thrift_proxy.protocols"
      },
      {
       "name": "binary",
       "category": "envoy.thrift_proxy.protocols"
      },
      {
       "name": "binary/non-strict",
       "category": "envoy.thrift_proxy.protocols"
      },
      {
       "name": "compact",
       "category": "envoy.thrift_proxy.protocols"
      },
      {
       "name": "twitter",
       "category": "envoy.thrift_proxy.protocols"
      },
      {
       "name": "envoy.watchdog.abort_action",
       "category": "envoy.guarddog_actions",
       "type_urls": [
        "envoy.watchdog.v3.AbortActionConfig"
       ]
      },
      {
       "name": "envoy.watchdog.profile_action",
       "category": "envoy.guarddog_actions",
       "type_urls": [
        "envoy.extensions.watchdog.profile_action.v3.ProfileActionConfig"
       ]
      },
      {
       "name": "envoy.quic.deterministic_connection_id_generator",
       "category": "envoy.quic.connection_id_generator",
       "type_urls": [
        "envoy.extensions.quic.connection_id_generator.v3.DeterministicConnectionIdGeneratorConfig"
       ]
      },
      {
       "name": "envoy.health_check.event_sink.file",
       "category": "envoy.health_check.event_sinks",
       "type_urls": [
        "envoy.extensions.health_check.event_sinks.file.v3.HealthCheckEventFileSink"
       ]
      },
      {
       "name": "envoy.rbac.matchers.upstream_ip_port",
       "category": "envoy.rbac.matchers",
       "type_urls": [
        "envoy.extensions.rbac.matchers.upstream_ip_port.v3.UpstreamIpPortMatcher"
       ]
      },
      {
       "name": "envoy.filters.udp.dns_filter",
       "category": "envoy.filters.udp_listener",
       "type_urls": [
        "envoy.extensions.filters.udp.dns_filter.v3.DnsFilterConfig"
       ]
      },
      {
       "name": "envoy.filters.udp_listener.udp_proxy",
       "category": "envoy.filters.udp_listener",
       "type_urls": [
        "envoy.extensions.filters.udp.udp_proxy.v3.UdpProxyConfig"
       ]
      },
      {
       "name": "envoy.http.early_header_mutation.header_mutation",
       "category": "envoy.http.early_header_mutation",
       "type_urls": [
        "envoy.extensions.http.early_header_mutation.header_mutation.v3.HeaderMutation"
       ]
      },
      {
       "name": "envoy.tls.cert_validator.default",
       "category": "envoy.tls.cert_validator"
      },
      {
       "name": "envoy.tls.cert_validator.spiffe",
       "category": "envoy.tls.cert_validator"
      },
      {
       "name": "envoy.listener_manager_impl.default",
       "category": "envoy.listener_manager_impl",
       "type_urls": [
        "envoy.config.listener.v3.ListenerManager"
       ]
      },
      {
       "name": "envoy.listener_manager_impl.validation",
       "category": "envoy.listener_manager_impl",
       "type_urls": [
        "envoy.config.listener.v3.ValidationListenerManager"
       ]
      },
      {
       "name": "envoy.config_subscription.ads",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.ads_collection",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.aggregated_grpc_collection",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.delta_grpc",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.delta_grpc_collection",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.filesystem",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.filesystem_collection",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.grpc",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.config_subscription.rest",
       "category": "envoy.config_subscription"
      },
      {
       "name": "envoy.buffer",
       "category": "envoy.filters.http.upstream"
      },
      {
       "name": "envoy.filters.http.admission_control",
       "category": "envoy.filters.http.upstream",
       "type_urls": [
        "envoy.extensions.filters.http.admission_control.v3.AdmissionControl"
       ]
      },
      {
       "name": "envoy.filters.http.buffer",
       "category": "envoy.filters.http.upstream",
       "type_urls": [
        "envoy.extensions.filters.http.buffer.v3.Buffer",
        "envoy.extensions.filters.http.buffer.v3.BufferPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.header_mutation",
       "category": "envoy.filters.http.upstream",
       "type_urls": [
        "envoy.extensions.filters.http.header_mutation.v3.HeaderMutation",
        "envoy.extensions.filters.http.header_mutation.v3.HeaderMutationPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.upstream_codec",
       "category": "envoy.filters.http.upstream",
       "type_urls": [
        "envoy.extensions.filters.http.upstream_codec.v3.UpstreamCodec"
       ]
      },
      {
       "name": "envoy.retry_priorities.previous_priorities",
       "category": "envoy.retry_priorities",
       "type_urls": [
        "envoy.extensions.retry.priority.previous_priorities.v3.PreviousPrioritiesConfig"
       ]
      },
      {
       "name": "envoy.matching.inputs.application_protocol",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.ApplicationProtocolInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.destination_ip",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.DestinationIPInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.destination_port",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.DestinationPortInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.direct_source_ip",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.DirectSourceIPInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.dns_san",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.ssl.v3.DnsSanInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.filter_state",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.FilterStateInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.server_name",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.ServerNameInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.source_ip",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.SourceIPInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.source_port",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.SourcePortInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.source_type",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.SourceTypeInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.subject",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.ssl.v3.SubjectInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.transport_protocol",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.network.v3.TransportProtocolInput"
       ]
      },
      {
       "name": "envoy.matching.inputs.uri_san",
       "category": "envoy.matching.network.input",
       "type_urls": [
        "envoy.extensions.matching.common_inputs.ssl.v3.UriSanInput"
       ]
      },
      {
       "name": "envoy.echo",
       "category": "envoy.filters.network"
      },
      {
       "name": "envoy.ext_authz",
       "category": "envoy.filters.network"
      },
      {
       "name": "envoy.filters.network.connection_limit",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.connection_limit.v3.ConnectionLimit"
       ]
      },
      {
       "name": "envoy.filters.network.direct_response",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.direct_response.v3.Config"
       ]
      },
      {
       "name": "envoy.filters.network.dubbo_proxy",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.dubbo_proxy.v3.DubboProxy"
       ]
      },
      {
       "name": "envoy.filters.network.echo",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.echo.v3.Echo"
       ]
      },
      {
       "name": "envoy.filters.network.ext_authz",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.ext_authz.v3.ExtAuthz"
       ]
      },
      {
       "name": "envoy.filters.network.http_connection_manager",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
       ]
      },
      {
       "name": "envoy.filters.network.local_ratelimit",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.local_ratelimit.v3.LocalRateLimit"
       ]
      },
      {
       "name": "envoy.filters.network.mongo_proxy",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.mongo_proxy.v3.MongoProxy"
       ]
      },
      {
       "name": "envoy.filters.network.ratelimit",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.ratelimit.v3.RateLimit"
       ]
      },
      {
       "name": "envoy.filters.network.rbac",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.rbac.v3.RBAC"
       ]
      },
      {
       "name": "envoy.filters.network.redis_proxy",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.redis_proxy.v3.RedisProxy"
       ]
      },
      {
       "name": "envoy.filters.network.sni_cluster",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.sni_cluster.v3.SniCluster"
       ]
      },
      {
       "name": "envoy.filters.network.sni_dynamic_forward_proxy",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.sni_dynamic_forward_proxy.v3.FilterConfig"
       ]
      },
      {
       "name": "envoy.filters.network.tcp_proxy",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy"
       ]
      },
      {
       "name": "envoy.filters.network.thrift_proxy",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.thrift_proxy.v3.ThriftProxy"
       ]
      },
      {
       "name": "envoy.filters.network.wasm",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.wasm.v3.Wasm"
       ]
      },
      {
       "name": "envoy.filters.network.zookeeper_proxy",
       "category": "envoy.filters.network",
       "type_urls": [
        "envoy.extensions.filters.network.zookeeper_proxy.v3.ZooKeeperProxy"
       ]
      },
      {
       "name": "envoy.http_connection_manager",
       "category": "envoy.filters.network"
      },
      {
       "name": "envoy.mongo_proxy",
       "category": "envoy.filters.network"
      },
      {
       "name": "envoy.ratelimit",
       "category": "envoy.filters.network"
      },
      {
       "name": "envoy.redis_proxy",
       "category": "envoy.filters.network"
      },
      {
       "name": "envoy.tcp_proxy",
       "category": "envoy.filters.network"
      },
      {
       "name": "envoy.access_loggers.file",
       "category": "envoy.access_loggers",
       "type_urls": [
        "envoy.extensions.access_loggers.file.v3.FileAccessLog"
       ]
      },
      {
       "name": "envoy.access_loggers.http_grpc",
       "category": "envoy.access_loggers",
       "type_urls": [
        "envoy.extensions.access_loggers.grpc.v3.HttpGrpcAccessLogConfig"
       ]
      },
      {
       "name": "envoy.access_loggers.open_telemetry",
       "category": "envoy.access_loggers",
       "type_urls": [
        "envoy.extensions.access_loggers.open_telemetry.v3.OpenTelemetryAccessLogConfig"
       ]
      },
      {
       "name": "envoy.access_loggers.stderr",
       "category": "envoy.access_loggers",
       "type_urls": [
        "envoy.extensions.access_loggers.stream.v3.StderrAccessLog"
       ]
      },
      {
       "name": "envoy.access_loggers.stdout",
       "category": "envoy.access_loggers",
       "type_urls": [
        "envoy.extensions.access_loggers.stream.v3.StdoutAccessLog"
       ]
      },
      {
       "name": "envoy.access_loggers.tcp_grpc",
       "category": "envoy.access_loggers",
       "type_urls": [
        "envoy.extensions.access_loggers.grpc.v3.TcpGrpcAccessLogConfig"
       ]
      },
      {
       "name": "envoy.access_loggers.wasm",
       "category": "envoy.access_loggers",
       "type_urls": [
        "envoy.extensions.access_loggers.wasm.v3.WasmAccessLog"
       ]
      },
      {
       "name": "envoy.file_access_log",
       "category": "envoy.access_loggers"
      },
      {
       "name": "envoy.http_grpc_access_log",
       "category": "envoy.access_loggers"
      },
      {
       "name": "envoy.open_telemetry_access_log",
       "category": "envoy.access_loggers"
      },
      {
       "name": "envoy.stderr_access_log",
       "category": "envoy.access_loggers"
      },
      {
       "name": "envoy.stdout_access_log",
       "category": "envoy.access_loggers"
      },
      {
       "name": "envoy.tcp_grpc_access_log",
       "category": "envoy.access_loggers"
      },
      {
       "name": "envoy.wasm_access_log",
       "category": "envoy.access_loggers"
      },
      {
       "name": "envoy.wasm.runtime.null",
       "category": "envoy.wasm.runtime"
      },
      {
       "name": "envoy.wasm.runtime.v8",
       "category": "envoy.wasm.runtime"
      },
      {
       "name": "envoy.udp_packet_writer.default",
       "category": "envoy.udp_packet_writer",
       "type_urls": [
        "envoy.extensions.udp_packet_writer.v3.UdpDefaultWriterFactory"
       ]
      },
      {
       "name": "envoy.udp_packet_writer.gso",
       "category": "envoy.udp_packet_writer",
       "type_urls": [
        "envoy.extensions.udp_packet_writer.v3.UdpGsoBatchWriterFactory"
       ]
      },
      {
       "name": "envoy.bandwidth_limit",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.buffer",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.cors",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.csrf",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.ext_authz",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.ext_proc",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.fault",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.filters.http.adaptive_concurrency",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.adaptive_concurrency.v3.AdaptiveConcurrency"
       ]
      },
      {
       "name": "envoy.filters.http.admission_control",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.admission_control.v3.AdmissionControl"
       ]
      },
      {
       "name": "envoy.filters.http.alternate_protocols_cache",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.alternate_protocols_cache.v3.FilterConfig"
       ]
      },
      {
       "name": "envoy.filters.http.aws_lambda",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.aws_lambda.v3.Config",
        "envoy.extensions.filters.http.aws_lambda.v3.PerRouteConfig"
       ]
      },
      {
       "name": "envoy.filters.http.aws_request_signing",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.aws_request_signing.v3.AwsRequestSigning",
        "envoy.extensions.filters.http.aws_request_signing.v3.AwsRequestSigningPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.bandwidth_limit",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.bandwidth_limit.v3.BandwidthLimit"
       ]
      },
      {
       "name": "envoy.filters.http.buffer",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.buffer.v3.Buffer",
        "envoy.extensions.filters.http.buffer.v3.BufferPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.cache",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.cache.v3.CacheConfig"
       ]
      },
      {
       "name": "envoy.filters.http.cdn_loop",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.cdn_loop.v3.CdnLoopConfig"
       ]
      },
      {
       "name": "envoy.filters.http.composite",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.composite.v3.Composite"
       ]
      },
      {
       "name": "envoy.filters.http.compressor",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.compressor.v3.Compressor",
        "envoy.extensions.filters.http.compressor.v3.CompressorPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.connect_grpc_bridge",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.connect_grpc_bridge.v3.FilterConfig"
       ]
      },
      {
       "name": "envoy.filters.http.cors",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.cors.v3.Cors",
        "envoy.extensions.filters.http.cors.v3.CorsPolicy"
       ]
      },
      {
       "name": "envoy.filters.http.csrf",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.csrf.v3.CsrfPolicy"
       ]
      },
      {
       "name": "envoy.filters.http.custom_response",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.custom_response.v3.CustomResponse"
       ]
      },
      {
       "name": "envoy.filters.http.decompressor",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.decompressor.v3.Decompressor"
       ]
      },
      {
       "name": "envoy.filters.http.dynamic_forward_proxy",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig",
        "envoy.extensions.filters.http.dynamic_forward_proxy.v3.PerRouteConfig"
       ]
      },
      {
       "name": "envoy.filters.http.ext_authz",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.ext_authz.v3.ExtAuthz",
        "envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.ext_proc",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.ext_proc.v3.ExtProcPerRoute",
        "envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor"
       ]
      },
      {
       "name": "envoy.filters.http.fault",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.fault.v3.HTTPFault"
       ]
      },
      {
       "name": "envoy.filters.http.file_system_buffer",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.file_system_buffer.v3.FileSystemBufferFilterConfig"
       ]
      },
      {
       "name": "envoy.filters.http.gcp_authn",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.gcp_authn.v3.GcpAuthnFilterConfig"
       ]
      },
      {
       "name": "envoy.filters.http.geoip",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.geoip.v3.Geoip"
       ]
      },
      {
       "name": "envoy.filters.http.grpc_field_extraction",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.grpc_field_extraction.v3.GrpcFieldExtractionConfig"
       ]
      },
      {
       "name": "envoy.filters.http.grpc_http1_bridge",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.grpc_http1_bridge.v3.Config"
       ]
      },
      {
       "name": "envoy.filters.http.grpc_http1_reverse_bridge",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.grpc_http1_reverse_bridge.v3.FilterConfig",
        "envoy.extensions.filters.http.grpc_http1_reverse_bridge.v3.FilterConfigPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.grpc_json_transcoder",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder"
       ]
      },
      {
       "name": "envoy.filters.http.grpc_stats",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.grpc_stats.v3.FilterConfig"
       ]
      },
      {
       "name": "envoy.filters.http.grpc_web",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.grpc_web.v3.GrpcWeb"
       ]
      },
      {
       "name": "envoy.filters.http.header_mutation",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.header_mutation.v3.HeaderMutation",
        "envoy.extensions.filters.http.header_mutation.v3.HeaderMutationPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.header_to_metadata",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.header_to_metadata.v3.Config"
       ]
      },
      {
       "name": "envoy.filters.http.health_check",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.health_check.v3.HealthCheck"
       ]
      },
      {
       "name": "envoy.filters.http.ip_tagging",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.ip_tagging.v3.IPTagging"
       ]
      },
      {
       "name": "envoy.filters.http.jwt_authn",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication",
        "envoy.extensions.filters.http.jwt_authn.v3.PerRouteConfig"
       ]
      },
      {
       "name": "envoy.filters.http.local_ratelimit",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit"
       ]
      },
      {
       "name": "envoy.filters.http.lua",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.lua.v3.Lua",
        "envoy.extensions.filters.http.lua.v3.LuaPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.match_delegate",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.common.matching.v3.ExtensionWithMatcher",
        "envoy.extensions.common.matching.v3.ExtensionWithMatcherPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.oauth2",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.oauth2.v3.OAuth2"
       ]
      },
      {
       "name": "envoy.filters.http.on_demand",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.on_demand.v3.OnDemand",
        "envoy.extensions.filters.http.on_demand.v3.PerRouteConfig"
       ]
      },
      {
       "name": "envoy.filters.http.original_src",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.original_src.v3.OriginalSrc"
       ]
      },
      {
       "name": "envoy.filters.http.rate_limit_quota",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.rate_limit_quota.v3.RateLimitQuotaFilterConfig",
        "envoy.extensions.filters.http.rate_limit_quota.v3.RateLimitQuotaOverride"
       ]
      },
      {
       "name": "envoy.filters.http.ratelimit",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.ratelimit.v3.RateLimit",
        "envoy.extensions.filters.http.ratelimit.v3.RateLimitPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.rbac",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.rbac.v3.RBAC",
        "envoy.extensions.filters.http.rbac.v3.RBACPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.router",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.router.v3.Router"
       ]
      },
      {
       "name": "envoy.filters.http.set_metadata",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.set_metadata.v3.Config"
       ]
      },
      {
       "name": "envoy.filters.http.stateful_session",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.stateful_session.v3.StatefulSession",
        "envoy.extensions.filters.http.stateful_session.v3.StatefulSessionPerRoute"
       ]
      },
      {
       "name": "envoy.filters.http.tap",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.tap.v3.Tap"
       ]
      },
      {
       "name": "envoy.filters.http.wasm",
       "category": "envoy.filters.http",
       "type_urls": [
        "envoy.extensions.filters.http.wasm.v3.Wasm"
       ]
      },
      {
       "name": "envoy.geoip",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.grpc_http1_bridge",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.grpc_json_transcoder",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.grpc_web",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.health_check",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.ip_tagging",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.local_rate_limit",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.lua",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.rate_limit",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.router",
       "category": "envoy.filters.http"
      },
      {
       "name": "envoy.internal_redirect_predicates.allow_listed_routes",
       "category": "envoy.internal_redirect_predicates",
       "type_urls": [
        "envoy.extensions.internal_redirect.allow_listed_routes.v3.AllowListedRoutesConfig"
       ]
      },
      {
       "name": "envoy.internal_redirect_predicates.previous_routes",
       "category": "envoy.internal_redirect_predicates",
       "type_urls": [
        "envoy.extensions.internal_redirect.previous_routes.v3.PreviousRoutesConfig"
       ]
      },
      {
       "name": "envoy.internal_redirect_predicates.safe_cross_scheme",
       "category": "envoy.internal_redirect_predicates",
       "type_urls": [
        "envoy.extensions.internal_redirect.safe_cross_scheme.v3.SafeCrossSchemeConfig"
       ]
      },
      {
       "name": "envoy.connection_handler.default",
       "category": "envoy.connection_handler"
      },
      {
       "name": "envoy.resource_monitors.fixed_heap",
       "category": "envoy.resource_monitors",
       "type_urls": [
        "envoy.extensions.resource_monitors.fixed_heap.v3.FixedHeapConfig"
       ]
      },
      {
       "name": "envoy.resource_monitors.injected_resource",
       "category": "envoy.resource_monitors",
       "type_urls": [
        "envoy.extensions.resource_monitors.injected_resource.v3.InjectedResourceConfig"
       ]
      },
      {
       "name": "envoy.retry_host_predicates.omit_canary_hosts",
       "category": "envoy.retry_host_predicates",
       "type_urls": [
        "envoy.extensions.retry.host.omit_canary_hosts.v3.OmitCanaryHostsPredicate"
       ]
      },
      {
       "name": "envoy.retry_host_predicates.omit_host_metadata",
       "category": "envoy.retry_host_predicates",
       "type_urls": [
        "envoy.extensions.retry.host.omit_host_metadata.v3.OmitHostMetadataConfig"
       ]
      },
      {
       "name": "envoy.retry_host_predicates.previous_hosts",
       "category": "envoy.retry_host_predicates",
       "type_urls": [
        "envoy.extensions.retry.host.previous_hosts.v3.PreviousHostsPredicate"
       ]
      },
      {
       "name": "envoy.transport_sockets.alts",
       "category": "envoy.transport_sockets.downstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.alts.v3.Alts"
       ]
      },
      {
       "name": "envoy.transport_sockets.quic",
       "category": "envoy.transport_sockets.downstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.quic.v3.QuicDownstreamTransport"
       ]
      },
      {
       "name": "envoy.transport_sockets.raw_buffer",
       "category": "envoy.transport_sockets.downstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.raw_buffer.v3.RawBuffer"
       ]
      },
      {
       "name": "envoy.transport_sockets.starttls",
       "category": "envoy.transport_sockets.downstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.starttls.v3.StartTlsConfig"
       ]
      },
      {
       "name": "envoy.transport_sockets.tap",
       "category": "envoy.transport_sockets.downstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.tap.v3.Tap"
       ]
      },
      {
       "name": "envoy.transport_sockets.tcp_stats",
       "category": "envoy.transport_sockets.downstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.tcp_stats.v3.Config"
       ]
      },
      {
       "name": "envoy.transport_sockets.tls",
       "category": "envoy.transport_sockets.downstream",
       "type_urls": [
        "envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext"
       ]
      },
      {
       "name": "raw_buffer",
       "category": "envoy.transport_sockets.downstream"
      },
      {
       "name": "starttls",
       "category": "envoy.transport_sockets.downstream"
      },
      {
       "name": "tls",
       "category": "envoy.transport_sockets.downstream"
      },
      {
       "name": "envoy.request_id.uuid",
       "category": "envoy.request_id",
       "type_urls": [
        "envoy.extensions.request_id.uuid.v3.UuidRequestIdConfig"
       ]
      },
      {
       "name": "envoy.quic.crypto_stream.server.quiche",
       "category": "envoy.quic.server.crypto_stream",
       "type_urls": [
        "envoy.extensions.quic.crypto_stream.v3.CryptoServerStreamConfig"
       ]
      },
      {
       "name": "envoy.filters.dubbo.router",
       "category": "envoy.dubbo_proxy.filters",
       "type_urls": [
        "envoy.extensions.filters.network.dubbo_proxy.router.v3.Router"
       ]
      },
      {
       "name": "quic.http_server_connection.default",
       "category": "quic.http_server_connection"
      },
      {
       "name": "envoy.matching.matchers.cel_matcher",
       "category": "envoy.matching.input_matchers",
       "type_urls": [
        "xds.type.matcher.v3.CelMatcher"
       ]
      },
      {
       "name": "envoy.matching.matchers.consistent_hashing",
       "category": "envoy.matching.input_matchers",
       "type_urls": [
        "envoy.extensions.matching.input_matchers.consistent_hashing.v3.ConsistentHashing"
       ]
      },
      {
       "name": "envoy.matching.matchers.ip",
       "category": "envoy.matching.input_matchers",
       "type_urls": [
        "envoy.extensions.matching.input_matchers.ip.v3.Ip"
       ]
      },
      {
       "name": "envoy.matching.matchers.runtime_fraction",
       "category": "envoy.matching.input_matchers",
       "type_urls": [
        "envoy.extensions.matching.input_matchers.runtime_fraction.v3.RuntimeFraction"
       ]
      },
      {
       "name": "auto",
       "category": "envoy.thrift_proxy.transports"
      },
      {
       "name": "framed",
       "category": "envoy.thrift_proxy.transports"
      },
      {
       "name": "header",
       "category": "envoy.thrift_proxy.transports"
      },
      {
       "name": "unframed",
       "category": "envoy.thrift_proxy.transports"
      },
      {
       "name": "envoy.cluster.eds",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.cluster.logical_dns",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.cluster.original_dst",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.cluster.static",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.cluster.strict_dns",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.clusters.aggregate",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.clusters.dynamic_forward_proxy",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.clusters.redis",
       "category": "envoy.clusters"
      },
      {
       "name": "envoy.access_loggers.extension_filters.cel",
       "category": "envoy.access_loggers.extension_filters",
       "type_urls": [
        "envoy.extensions.access_loggers.filters.cel.v3.ExpressionFilter"
       ]
      },
      {
       "name": "envoy.key_value.file_based",
       "category": "envoy.common.key_value",
       "type_urls": [
        "envoy.extensions.key_value.file_based.v3.FileBasedKeyValueStoreConfig"
       ]
      },
      {
       "name": "envoy.extensions.http.custom_response.local_response_policy",
       "category": "envoy.http.custom_response",
       "type_urls": [
        "envoy.extensions.http.custom_response.local_response_policy.v3.LocalResponsePolicy"
       ]
      },
      {
       "name": "envoy.extensions.http.custom_response.redirect_policy",
       "category": "envoy.http.custom_response",
       "type_urls": [
        "envoy.extensions.http.custom_response.redirect_policy.v3.RedirectPolicy"
       ]
      },
      {
       "name": "envoy.matching.custom_matchers.trie_matcher",
       "category": "envoy.matching.http.custom_matchers",
       "type_urls": [
        "xds.type.matcher.v3.IPMatcher"
       ]
      },
      {
       "name": "default",
       "category": "network.connection.client"
      },
      {
       "name": "envoy_internal",
       "category": "network.connection.client"
      },
      {
       "name": "envoy.compression.brotli.decompressor",
       "category": "envoy.compression.decompressor",
       "type_urls": [
        "envoy.extensions.compression.brotli.decompressor.v3.Brotli"
       ]
      },
      {
       "name": "envoy.compression.gzip.decompressor",
       "category": "envoy.compression.decompressor",
       "type_urls": [
        "envoy.extensions.compression.gzip.decompressor.v3.Gzip"
       ]
      },
      {
       "name": "envoy.compression.zstd.decompressor",
       "category": "envoy.compression.decompressor",
       "type_urls": [
        "envoy.extensions.compression.zstd.decompressor.v3.Zstd"
       ]
      },
      {
       "name": "envoy.extensions.http.cache.file_system_http_cache",
       "category": "envoy.http.cache",
       "type_urls": [
        "envoy.extensions.http.cache.file_system_http_cache.v3.FileSystemHttpCacheConfig"
       ]
      },
      {
       "name": "envoy.extensions.http.cache.simple",
       "category": "envoy.http.cache",
       "type_urls": [
        "envoy.extensions.http.cache.simple_http_cache.v3.SimpleHttpCacheConfig"
       ]
      },
      {
       "name": "envoy.http.stateful_header_formatters.preserve_case",
       "category": "envoy.http.stateful_header_formatters",
       "type_urls": [
        "envoy.extensions.http.header_formatters.preserve_case.v3.PreserveCaseFormatterConfig"
       ]
      },
      {
       "name": "preserve_case",
       "category": "envoy.http.stateful_header_formatters"
      },
      {
       "name": "envoy.quic.proof_source.filter_chain",
       "category": "envoy.quic.proof_source",
       "type_urls": [
        "envoy.extensions.quic.proof_source.v3.ProofSourceConfig"
       ]
      },
      {
       "name": "quic.server_preferred_address.fixed",
       "category": "envoy.quic.server_preferred_address",
       "type_urls": [
        "envoy.extensions.quic.server_preferred_address.v3.FixedServerPreferredAddressConfig"
       ]
      },
      {
       "name": "envoy.path.match.uri_template.uri_template_matcher",
       "category": "envoy.path.match",
       "type_urls": [
        "envoy.extensions.path.match.uri_template.v3.UriTemplateMatchConfig"
       ]
      },
      {
       "name": "envoy.http.original_ip_detection.custom_header",
       "category": "envoy.http.original_ip_detection",
       "type_urls": [
        "envoy.extensions.http.original_ip_detection.custom_header.v3.CustomHeaderConfig"
       ]
      },
      {
       "name": "envoy.http.original_ip_detection.xff",
       "category": "envoy.http.original_ip_detection",
       "type_urls": [
        "envoy.extensions.http.original_ip_detection.xff.v3.XffConfig"
       ]
      },
      {
       "name": "envoy.config_mux.delta_grpc_mux_factory",
       "category": "envoy.config_mux"
      },
      {
       "name": "envoy.config_mux.grpc_mux_factory",
       "category": "envoy.config_mux"
      },
      {
       "name": "envoy.config_mux.new_grpc_mux_factory",
       "category": "envoy.config_mux"
      },
      {
       "name": "envoy.config_mux.sotw_grpc_mux_factory",
       "category": "envoy.config_mux"
      },
      {
       "name": "envoy.grpc_credentials.aws_iam",
       "category": "envoy.grpc_credentials"
      },
      {
       "name": "envoy.grpc_credentials.default",
       "category": "envoy.grpc_credentials"
      },
      {
       "name": "envoy.grpc_credentials.file_based_metadata",
       "category": "envoy.grpc_credentials"
      },
      {
       "name": "envoy.compression.brotli.compressor",
       "category": "envoy.compression.compressor",
       "type_urls": [
        "envoy.extensions.compression.brotli.compressor.v3.Brotli"
       ]
      },
      {
       "name": "envoy.compression.gzip.compressor",
       "category": "envoy.compression.compressor",
       "type_urls": [
        "envoy.extensions.compression.gzip.compressor.v3.Gzip"
       ]
      },
      {
       "name": "envoy.compression.zstd.compressor",
       "category": "envoy.compression.compressor",
       "type_urls": [
        "envoy.extensions.compression.zstd.compressor.v3.Zstd"
       ]
      },
      {
       "name": "envoy.regex_engines.google_re2",
       "category": "envoy.regex_engines",
       "type_urls": [
        "envoy.extensions.regex_engines.v3.GoogleRE2"
       ]
      },
      {
       "name": "envoy.http.stateful_session.cookie",
       "category": "envoy.http.stateful_session",
       "type_urls": [
        "envoy.extensions.http.stateful_session.cookie.v3.CookieBasedSessionState"
       ]
      },
      {
       "name": "envoy.http.stateful_session.header",
       "category": "envoy.http.stateful_session",
       "type_urls": [
        "envoy.extensions.http.stateful_session.header.v3.HeaderBasedSessionState"
       ]
      },
      {
       "name": "envoy.route.early_data_policy.default",
       "category": "envoy.route.early_data_policy",
       "type_urls": [
        "envoy.extensions.early_data.v3.DefaultEarlyDataPolicy"
       ]
      }
     ]
    },
    "static_resources": {
     "clusters": [
      {
       "name": "xds_cluster",
       "type": "STRICT_DNS",
       "load_assignment": {
        "cluster_name": "xds_cluster",
        "endpoints": [
         {
          "lb_endpoints": [
           {
            "endpoint": {
             "address": {
              "socket_address": {
               "address": "host.docker.internal",
               "port_value": 18000
              }
             }
            }
           }
          ]
         }
        ]
       },
       "typed_extension_protocol_options": {
        "envoy.extensions.upstreams.http.v3.HttpProtocolOptions": {
         "@type": "type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions",
         "explicit_http_config": {
          "http2_protocol_options": {}
         }
        }
       }
      }
     ]
    },
    "dynamic_resources": {
     "lds_config": {
      "ads": {},
      "resource_api_version": "V3"
     },
     "cds_config": {
      "ads": {},
      "resource_api_version": "V3"
     },
     "ads_config": {
      "api_type": "DELTA_GRPC",
      "grpc_services": [
       {
        "envoy_grpc": {
         "cluster_name": "xds_cluster"
        }
       }
      ],
      "transport_api_version": "V3"
     }
    },
    "cluster_manager": {
     "outlier_detection": {
      "event_log_path": "/dev/stdout"
     }
    },
    "admin": {
     "address": {
      "socket_address": {
       "address": "0.0.0.0",
       "port_value": 9901
      }
     }
    },
    "layered_runtime": {
     "layers": [
      {
       "name": "static_layer_0",
       "static_layer": {
        "envoy.reloadable_features.successful_active_health_check_uneject_host": false
       }
      }
     ]
    }
   },
   "last_updated": "2023-11-29T18:15:12.426Z"
  },
  {
   "@type": "type.googleapis.com/envoy.admin.v3.ClustersConfigDump",
   "version_info": "2023-11-29T15:15:17-03:00",
   "static_clusters": [
    {
     "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "xds_cluster",
      "type": "STRICT_DNS",
      "load_assignment": {
       "cluster_name": "xds_cluster",
       "endpoints": [
        {
         "lb_endpoints": [
          {
           "endpoint": {
            "address": {
             "socket_address": {
              "address": "host.docker.internal",
              "port_value": 18000
             }
            }
           }
          }
         ]
        }
       ]
      },
      "typed_extension_protocol_options": {
       "envoy.extensions.upstreams.http.v3.HttpProtocolOptions": {
        "@type": "type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions",
        "explicit_http_config": {
         "http2_protocol_options": {}
        }
       }
      }
     },
     "last_updated": "2023-11-29T18:15:12.448Z"
    }
   ],
   "dynamic_active_clusters": [
    {
     "version_info": "4c29bced2492cb08fcb34b31886c68bdf73e9e739b902e7d8c09335d97ad96e4",
     "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "sample-shard",
      "type": "STRICT_DNS",
      "health_checks": [
       {
        "timeout": "10s",
        "interval": "5s",
        "unhealthy_threshold": 5,
        "healthy_threshold": 2,
        "reuse_connection": false,
        "http_health_check": {
         "path": "/"
        },
        "no_traffic_interval": "5s",
        "event_log_path": "/dev/stdout",
        "always_log_health_check_failures": true
       }
      ],
      "circuit_breakers": {
       "thresholds": [
        {
         "max_connections": 1024,
         "max_pending_requests": 1024,
         "max_requests": 1024,
         "max_retries": 3
        }
       ]
      },
      "dns_lookup_family": "V4_ONLY",
      "outlier_detection": {
       "consecutive_5xx": 2,
       "interval": "1s",
       "base_ejection_time": "60s",
       "max_ejection_percent": 100,
       "max_ejection_time": "120s"
      },
      "common_lb_config": {
       "healthy_panic_threshold": {}
      },
      "load_assignment": {
       "cluster_name": "sample-shard",
       "endpoints": [
        {
         "locality": {
          "region": "sa-east-1",
          "zone": "az-a",
          "sub_zone": "cell-1"
         },
         "lb_endpoints": [
          {
           "endpoint": {
            "address": {
             "socket_address": {
              "address": "json-server-mock-cell1",
              "port_value": 80
             }
            },
            "health_check_config": {
             "port_value": 80,
             "hostname": "json-server-mock-cell1"
            }
           },
           "load_balancing_weight": 100
          }
         ],
         "priority": 1
        },
        {
         "locality": {
          "region": "sa-east-1",
          "zone": "az-b",
          "sub_zone": "cell-2"
         },
         "lb_endpoints": [
          {
           "endpoint": {
            "address": {
             "socket_address": {
              "address": "json-server-mock-cell2",
              "port_value": 80
             }
            },
            "health_check_config": {
             "port_value": 80,
             "hostname": "json-server-mock-cell2"
            }
           },
           "load_balancing_weight": 100
          }
         ],
         "priority": 1
        }
       ]
      }
     },
     "last_updated": "2023-11-29T18:15:22.448Z"
    },
    {
     "version_info": "309870cc826e1cd0dff766a0768367f5f6dd758bd3accbe244a2a26fc25b0b92",
     "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "shard-2",
      "type": "STRICT_DNS",
      "health_checks": [
       {
        "timeout": "5s",
        "interval": "5s",
        "unhealthy_threshold": 6,
        "healthy_threshold": 2,
        "reuse_connection": false,
        "http_health_check": {
         "path": "/"
        },
        "no_traffic_interval": "5s",
        "event_log_path": "/dev/stdout",
        "always_log_health_check_failures": true
       }
      ],
      "circuit_breakers": {
       "thresholds": [
        {
         "max_connections": 1024,
         "max_pending_requests": 1024,
         "max_requests": 1024,
         "max_retries": 3
        }
       ]
      },
      "dns_lookup_family": "V4_ONLY",
      "outlier_detection": {
       "consecutive_5xx": 2,
       "interval": "1s",
       "base_ejection_time": "60s",
       "max_ejection_percent": 100,
       "max_ejection_time": "120s"
      },
      "common_lb_config": {
       "healthy_panic_threshold": {}
      },
      "load_assignment": {
       "cluster_name": "shard-2",
       "endpoints": [
        {
         "locality": {
          "region": "sa-east-1",
          "zone": "az-b",
          "sub_zone": "cell-3"
         },
         "lb_endpoints": [
          {
           "endpoint": {
            "address": {
             "socket_address": {
              "address": "json-server-mock-cell3",
              "port_value": 80
             }
            },
            "health_check_config": {
             "port_value": 80,
             "hostname": "json-server-mock-cell3"
            }
           },
           "load_balancing_weight": 100
          }
         ],
         "priority": 1
        }
       ]
      }
     },
     "last_updated": "2023-11-29T18:15:22.450Z"
    }
   ]
  },
  {
   "@type": "type.googleapis.com/envoy.admin.v3.EndpointsConfigDump",
   "static_endpoint_configs": [
    {
     "endpoint_config": {
      "@type": "type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",
      "cluster_name": "xds_cluster",
      "endpoints": [
       {
        "locality": {},
        "lb_endpoints": [
         {
          "endpoint": {
           "address": {
            "socket_address": {
             "address": "192.168.65.254",
             "port_value": 18000
            }
           },
           "health_check_config": {},
           "hostname": "host.docker.internal"
          },
          "health_status": "HEALTHY",
          "metadata": {},
          "load_balancing_weight": 1
         }
        ]
       }
      ],
      "policy": {
       "overprovisioning_factor": 140
      }
     }
    }
   ],
   "dynamic_endpoint_configs": [
    {
     "endpoint_config": {
      "@type": "type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",
      "cluster_name": "shard-2",
      "endpoints": [
       {
        "locality": {
         "region": "sa-east-1",
         "zone": "az-b",
         "sub_zone": "cell-3"
        },
        "lb_endpoints": [
         {
          "endpoint": {
           "address": {
            "socket_address": {
             "address": "192.168.224.4",
             "port_value": 80
            }
           },
           "health_check_config": {
            "hostname": "json-server-mock-cell3"
           },
           "hostname": "json-server-mock-cell3"
          },
          "health_status": "HEALTHY",
          "metadata": {},
          "load_balancing_weight": 100
         }
        ],
        "priority": 1
       }
      ],
      "policy": {
       "overprovisioning_factor": 140
      }
     }
    },
    {
     "endpoint_config": {
      "@type": "type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",
      "cluster_name": "sample-shard",
      "endpoints": [
       {
        "locality": {
         "region": "sa-east-1",
         "zone": "az-a",
         "sub_zone": "cell-1"
        },
        "lb_endpoints": [
         {
          "endpoint": {
           "address": {
            "socket_address": {
             "address": "192.168.224.5",
             "port_value": 80
            }
           },
           "health_check_config": {
            "hostname": "json-server-mock-cell1"
           },
           "hostname": "json-server-mock-cell1"
          },
          "health_status": "HEALTHY",
          "metadata": {},
          "load_balancing_weight": 100
         }
        ],
        "priority": 1
       },
       {
        "locality": {
         "region": "sa-east-1",
         "zone": "az-b",
         "sub_zone": "cell-2"
        },
        "lb_endpoints": [
         {
          "endpoint": {
           "address": {
            "socket_address": {
             "address": "192.168.224.3",
             "port_value": 80
            }
           },
           "health_check_config": {
            "hostname": "json-server-mock-cell2"
           },
           "hostname": "json-server-mock-cell2"
          },
          "health_status": "HEALTHY",
          "metadata": {},
          "load_balancing_weight": 100
         }
        ],
        "priority": 1
       }
      ],
      "policy": {
       "overprovisioning_factor": 140
      }
     }
    }
   ]
  },
  {
   "@type": "type.googleapis.com/envoy.admin.v3.ListenersConfigDump",
   "version_info": "2023-11-29T15:15:17-03:00",
   "dynamic_listeners": [
    {
     "name": "listener_0",
     "active_state": {
      "version_info": "5e661ad6da647a0e5bc7485fd75e882a8a4d9c1a6fac17098f21327bb0c81f0e",
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "listener_0",
       "address": {
        "socket_address": {
         "address": "0.0.0.0",
         "port_value": 11000
        }
       },
       "filter_chains": [
        {
         "filters": [
          {
           "name": "envoy.filters.network.http_connection_manager",
           "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
            "stat_prefix": "envoy_listener_11000",
            "rds": {
             "config_source": {
              "api_config_source": {
               "api_type": "GRPC",
               "grpc_services": [
                {
                 "envoy_grpc": {
                  "cluster_name": "xds_cluster"
                 }
                }
               ],
               "set_node_on_first_message_only": true,
               "transport_api_version": "V3"
              },
              "resource_api_version": "V3"
             },
             "route_config_name": "router_config"
            },
            "http_filters": [
             {
              "name": "envoy.filters.http.router",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"
              }
             }
            ],
            "use_remote_address": true,
            "normalize_path": true,
            "merge_slashes": true,
            "common_http_protocol_options": {
             "idle_timeout": "3600s"
            },
            "path_with_escaped_slashes_action": "UNESCAPE_AND_REDIRECT"
           }
          }
         ]
        }
       ]
      },
      "last_updated": "2023-11-29T18:15:22.473Z"
     }
    }
   ]
  },
  {
   "@type": "type.googleapis.com/envoy.admin.v3.ScopedRoutesConfigDump"
  },
  {
   "@type": "type.googleapis.com/envoy.admin.v3.RoutesConfigDump",
   "dynamic_route_configs": [
    {
     "version_info": "2023-11-29T15:16:21-03:00",
     "route_config": {
      "@type": "type.googleapis.com/envoy.config.route.v3.RouteConfiguration",
      "name": "router_config",
      "virtual_hosts": [
       {
        "name": "vh_all",
        "domains": [
         "*"
        ],
        "routes": [
         {
          "match": {
           "prefix": "/",
           "headers": [
            {
             "name": "x_code",
             "string_match": {
              "safe_regex": {
               "google_re2": {},
               "regex": "^.*(1|2)$"
              }
             }
            }
           ]
          },
          "route": {
           "cluster": "shard-2",
           "auto_host_rewrite": true
          }
         },
         {
          "match": {
           "prefix": "/"
          },
          "route": {
           "cluster": "sample-shard",
           "auto_host_rewrite": true
          }
         }
        ]
       }
      ]
     },
     "last_updated": "2023-11-29T18:16:21.986Z"
    }
   ]
  },
  {
   "@type": "type.googleapis.com/envoy.admin.v3.SecretsConfigDump"
  }
 ]
}
cpakulski commented 11 months ago

Based on this config you should see that a host is ejected for 60 secs the first time and for 120 secs the next time and each subsequent ejection should be 120 secs (because it is capped at 120 secs). I recently changed logic in eject/uneject but that change was required for frequency-based OD, not consecutive failures. Can you change interval to say 30 secs and see if it helps. If not, I will try to replicate is and debug it.

leandrocurioso commented 11 months ago

OK I will try with 30s and return here.

leandrocurioso commented 11 months ago

@cpakulski

I tried with 0.27.1 and outlier logging enabled and as you can sse:

Changed base ejection to 30s and max ejection still 120s

2023-11-29 16:09:34 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:09:32.749Z","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"EJECT","num_ejections":1,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:10:04 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:10:03.234Z","secs_since_last_action":"30","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"UNEJECT","num_ejections":1,"enforced":false}
2023-11-29 16:10:14 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:10:06.755Z","secs_since_last_action":"3","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"EJECT","num_ejections":2,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:10:44 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:10:37.248Z","secs_since_last_action":"30","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"UNEJECT","num_ejections":2,"enforced":false}
2023-11-29 16:10:44 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:10:40.751Z","secs_since_last_action":"3","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"EJECT","num_ejections":3,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:11:14 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:11:11.260Z","secs_since_last_action":"30","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"UNEJECT","num_ejections":3,"enforced":false}
2023-11-29 16:11:14 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:11:13.750Z","secs_since_last_action":"2","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"EJECT","num_ejections":4,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:11:44 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:11:44.277Z","secs_since_last_action":"30","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"UNEJECT","num_ejections":4,"enforced":false}
2023-11-29 16:11:54 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:11:47.763Z","secs_since_last_action":"3","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"EJECT","num_ejections":5,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:12:24 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:12:18.289Z","secs_since_last_action":"30","cluster_name":"sample-shard","upstream_url":"[172.19.0.5:80](http://172.19.0.5/)","action":"UNEJECT","num_ejections":5,"enforced":false}

The time is not being multiplied for each ejection (look at the timestamp and secs_since_last_action property) it is always capped by the base_ejection (really strange),

I also tried with these versions and the behaviour is the same. https://github.com/envoyproxy/envoy/issues/31070#issuecomment-1830689747

Maybe a bug?

cpakulski commented 11 months ago

Thanks for checking it. Just to make sure, the last logs were captured with the following config:

 "consecutive_5xx": 2,
       "interval": "30s",
       "base_ejection_time": "60s",
       "max_ejection_percent": 100,
       "max_ejection_time": "120s"

correct?

leandrocurioso commented 11 months ago

@cpakulski

only difference is that the interval is 1s

cpakulski commented 11 months ago

OK. Can you change interval to 30 secs? I suspect that if you have sporadic traffic and no errors are reported within that 1 second, OD will decrement backoff multiplier. Can you make it longer, so you are sure that when host is UNEJECTED, next failure will be within "interval" period after being UNEJECTED.

leandrocurioso commented 11 months ago

Okay I will try and return here.

leandrocurioso commented 11 months ago

@cpakulski

For that config:

"outlier_detection": {
     "consecutive_5xx": 2,
     "interval": "30s",
     "base_ejection_time": "30s",
     "max_ejection_percent": 100,
     "max_ejection_time": "120s"
}

The logs:

2023-11-29 16:36:48 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:36:44.041Z","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"EJECT","num_ejections":1,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:37:28 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:37:20.939Z","secs_since_last_action":"36","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"UNEJECT","num_ejections":1,"enforced":false}
2023-11-29 16:37:28 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:37:24.026Z","secs_since_last_action":"3","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"EJECT","num_ejections":2,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:38:58 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:38:50.946Z","secs_since_last_action":"86","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"UNEJECT","num_ejections":2,"enforced":false}
2023-11-29 16:38:58 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:38:53.413Z","secs_since_last_action":"2","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"EJECT","num_ejections":3,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:40:58 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:40:50.945Z","secs_since_last_action":"117","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"UNEJECT","num_ejections":3,"enforced":false}
2023-11-29 16:40:58 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:40:53.418Z","secs_since_last_action":"2","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"EJECT","num_ejections":4,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:43:28 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:43:20.951Z","secs_since_last_action":"147","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"UNEJECT","num_ejections":4,"enforced":false}
2023-11-29 16:43:28 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:43:23.411Z","secs_since_last_action":"2","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"EJECT","num_ejections":5,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:45:58 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:45:50.962Z","secs_since_last_action":"147","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"UNEJECT","num_ejections":5,"enforced":false}
2023-11-29 16:45:58 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:45:53.418Z","secs_since_last_action":"2","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"EJECT","num_ejections":6,"enforced":true,"eject_consecutive_event":{}}
2023-11-29 16:48:28 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:48:20.967Z","secs_since_last_action":"147","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"UNEJECT","num_ejections":6,"enforced":false}
2023-11-29 16:48:28 {"type":"CONSECUTIVE_5XX","timestamp":"2023-11-29T19:48:24.425Z","secs_since_last_action":"3","cluster_name":"sample-shard","upstream_url":"[172.21.0.2:80](http://172.21.0.2/)","action":"EJECT","num_ejections":7,"enforced":true,"eject_consecutive_event":{}}

Now secs_since_last_action property is being increased and the multiplication is happening! Very nice!

But one question what is the minimum interval that I should use? And if an user sets 1s interval is considered a bug?l

cpakulski commented 11 months ago

It is not a bug for 1 sec interval. If you generate more than 2 requests per second, when node is un-ejected next failure will happen within "interval" time of 1 secs and you will see incremental backoff-time. If you do not have enough traffic within that 1 secs, OD algorithm thinks "aha, interval passed and I did not see a failure, so probably host is healthy now" and decrements backoff multiplier. For consecutive errors OD, you can make interval equal to base_ejection_time, because a host is ejected immediately after threshold of consecutive errors is reached. In other OD methods (success rate, frequency) ejection happens only when "interval" timer fires, so you may want to have it shorter, but again for consecutive errors I think it buys you nothing.

leandrocurioso commented 11 months ago

@cpakulski thanks a lot for the explanation!