search

hashicorp / consul-k8s

First-class support for Consul Service Mesh on Kubernetes
https://www.consul.io/docs/k8s
Mozilla Public License 2.0
670 stars 324 forks source link

Tracing not working after upgrade from 1.13.x to 1.14.2 #1850

Closed nascimento closed 1 year ago

nascimento commented 1 year ago

Community Note

Overview of the Issue

After upgrade consul from 1.13 to 1.14.2 tracing stopped working.

Reproduction Steps

Steps to reproduce this issue, eg:

  1. When running helm install with the following proxydefaults.yml: 
    apiVersion: consul.hashicorp.com/v1alpha1
kind: ProxyDefaults
metadata:
name: global
spec:
meshGateway:
mode: local
config:
protocol: http
envoy_tracing_json: |
  {
    "http": {
      "name": "envoy.tracers.zipkin",
      "typedConfig": {
        "@type": "type.googleapis.com/envoy.config.trace.v3.ZipkinConfig",
        "collector_cluster": "zipkin",
        "collector_endpoint_version": "HTTP_JSON",
        "collector_endpoint": "/api/v2/spans",
        "shared_span_context": false
      }
    }
  }
envoy_extra_static_clusters_json: |
  {
    "name": "zipkin",
    "type": "STRICT_DNS",
    "connect_timeout": "3s",
    "dns_lookup_family": "V4_ONLY",
    "lb_policy": "ROUND_ROBIN",
    "load_assignment": {
      "cluster_name": "zipkin",
      "endpoints": [
        {
          "lb_endpoints": [
            {
              "endpoint": {
                "address": {
                  "socket_address": {
                    "address": "jaeger-collector.observability-system",
                    "port_value": 9411,
                    "protocol": "TCP"
                  }
                }
              }
            }
          ]
        }
      ]
    }
  }

No errors found.

Logs

consul-dataplane log debug:

Click me ``` 2023-01-23T22:35:24.015Z [INFO] consul-dataplane: started consul-dataplane process 2023-01-23T22:35:24.016Z [INFO] consul-dataplane.server-connection-manager: trying to connect to a Consul server 2023-01-23T22:35:24.020Z [DEBUG] consul-dataplane.server-connection-manager: Resolved DNS name: name=consul-server.gateway-system.svc ip-addrs=["{MASK_IP.191.33 }", "{MASK_IP.55.146 }", "{MASK_IP.92.83 }"] 2023-01-23T22:35:24.021Z [INFO] consul-dataplane.server-connection-manager: discovered Consul servers: addresses=[MASK_IP.191.33:8502, MASK_IP.55.146:8502, MASK_IP.92.83:8502] 2023-01-23T22:35:24.021Z [INFO] consul-dataplane.server-connection-manager: current prioritized list of known Consul servers: addresses=[MASK_IP.92.83:8502, MASK_IP.191.33:8502, MASK_IP.55.146:8502] 2023-01-23T22:35:24.021Z [DEBUG] consul-dataplane.server-connection-manager: switching to Consul server: address=MASK_IP.92.83:8502 2023-01-23T22:35:24.222Z [DEBUG] consul-dataplane.server-connection-manager: switched to Consul server successfully: address=MASK_IP.92.83:8502 2023-01-23T22:35:24.223Z [DEBUG] consul-dataplane.server-connection-manager: feature: supported=true name=DATAPLANE_FEATURES_EDGE_CERTIFICATE_MANAGEMENT 2023-01-23T22:35:24.223Z [DEBUG] consul-dataplane.server-connection-manager: feature: supported=true name=DATAPLANE_FEATURES_ENVOY_BOOTSTRAP_CONFIGURATION 2023-01-23T22:35:24.223Z [DEBUG] consul-dataplane.server-connection-manager: feature: supported=true name=DATAPLANE_FEATURES_WATCH_SERVERS 2023-01-23T22:35:24.223Z [INFO] consul-dataplane.server-connection-manager: connected to Consul server: address=MASK_IP.92.83:8502 2023-01-23T22:35:24.223Z [INFO] consul-dataplane: connected to Consul server over gRPC: initial_server_address=MASK_IP.92.83:8502 2023-01-23T22:35:24.223Z [INFO] consul-dataplane: starting envoy xDS server: address=127.0.0.1:46559 2023-01-23T22:35:24.224Z [INFO] consul-dataplane.server-connection-manager: updated known Consul servers from watch stream: addresses=[MASK_IP.191.33:8502, MASK_IP.55.146:8502, MASK_IP.92.83:8502] 2023-01-23T22:35:24.226Z [DEBUG] consul-dataplane: generated envoy bootstrap config: config= | { | "admin": { | "access_log_path": "/dev/null", | "address": { | "socket_address": { | "address": "127.0.0.1", | "port_value": 19000 | } | } | }, | "node": { | "cluster": "itau-service-provider", | "id": "itau-service-provider-846d58c48b-j97jn-itau-service-provider-sidecar-proxy", | "metadata": { | "node_name": "ip-10-147-139-71.ec2.internal-virtual", | "namespace": "default", | "partition": "default" | } | }, | "layered_runtime": { | "layers": [ | { | "name": "base", | "static_layer": { | "re2.max_program_size.error_level": 1048576 | } | } | ] | }, | "static_resources": { | "clusters": [ | { | "name": "consul-dataplane", | "ignore_health_on_host_removal": false, | "connect_timeout": "1s", | "type": "STATIC", | "http2_protocol_options": {}, | "loadAssignment": { | "clusterName": "consul-dataplane", | "endpoints": [ | { | "lbEndpoints": [ | { | "endpoint": { | "address": { | "socket_address": { | "address": "127.0.0.1", | "port_value": 46559 | } | } | } | } | ] | } | ] | } | }, | { | "name": "zipkin", | "type": "STRICT_DNS", | "connect_timeout": "3s", | "dns_lookup_family": "V4_ONLY", | "lb_policy": "ROUND_ROBIN", | "load_assignment": { | "cluster_name": "zipkin", | "endpoints": [ | { | "lb_endpoints": [ | { | "endpoint": { | "address": { | "socket_address": { | "address": "jaeger-collector.observability-system", | "port_value": 9411, | "protocol": "TCP" | } | } | } | } | ] | } | ] | } | }, | { | "name": "prometheus_backend", | "ignore_health_on_host_removal": false, | "connect_timeout": "5s", | "type": "STATIC", | "http_protocol_options": {}, | "loadAssignment": { | "clusterName": "prometheus_backend", | "endpoints": [ | { | "lbEndpoints": [ | { | "endpoint": { | "address": { | "socket_address": { | "address": "127.0.0.1", | "port_value": 20100 | } | } | } | } | ] | } | ] | } | } | ], | "listeners": [ | { | "name": "envoy_prometheus_metrics_listener", | "address": { | "socket_address": { | "address": "0.0.0.0", | "port_value": 20200 | } | }, | "filter_chains": [ | { | "filters": [ | { | "name": "envoy.filters.network.http_connection_manager", | "typedConfig": { | "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager", | "stat_prefix": "envoy_prometheus_metrics", | "codec_type": "HTTP1", | "route_config": { | "name": "self_admin_route", | "virtual_hosts": [ | { | "name": "self_admin", | "domains": [ | "*" | ], | "routes": [ | { | "match": { | "path": "/metrics" | }, | "route": { | "cluster": "prometheus_backend", | "prefix_rewrite": "/stats/prometheus" | } | }, | { | "match": { | "prefix": "/" | }, | "direct_response": { | "status": 404 | } | } | ] | } | ] | }, | "http_filters": [ | { | "name": "envoy.filters.http.router", | "typedConfig": { | "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router" | } | } | ] | } | } | ] | } | ] | } | ] | }, | "stats_config": { | "stats_tags": [ | { | "regex": "^cluster\\.(?:passthrough~)?((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.custom_hash" | }, | { | "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.service_subset" | }, | { | "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.service" | }, | { | "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.namespace" | }, | { | "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:([^.]+)\\.)?[^.]+\\.internal[^.]*\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.partition" | }, | { | "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?([^.]+)\\.internal[^.]*\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.datacenter" | }, | { | "regex": "^cluster\\.([^.]+\\.(?:[^.]+\\.)?([^.]+)\\.external\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.peer" | }, | { | "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.routing_type" | }, | { | "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)", | "tag_name": "consul.destination.trust_domain" | }, | { | "regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", | "tag_name": "consul.destination.target" | }, | { | "regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)", | "tag_name": "consul.destination.full_target" | }, | { | "regex": "^(?:tcp|http)\\.upstream(?:_peered)?\\.(([^.]+)(?:\\.[^.]+)?(?:\\.[^.]+)?\\.[^.]+\\.)", | "tag_name": "consul.upstream.service" | }, | { | "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?(?:\\.[^.]+)?\\.([^.]+)\\.)", | "tag_name": "consul.upstream.datacenter" | }, | { | "regex": "^(?:tcp|http)\\.upstream_peered\\.([^.]+(?:\\.[^.]+)?\\.([^.]+)\\.)", | "tag_name": "consul.upstream.peer" | }, | { | "regex": "^(?:tcp|http)\\.upstream(?:_peered)?\\.([^.]+(?:\\.([^.]+))?(?:\\.[^.]+)?\\.[^.]+\\.)", | "tag_name": "consul.upstream.namespace" | }, | { | "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?(?:\\.([^.]+))?\\.[^.]+\\.)", | "tag_name": "consul.upstream.partition" | }, | { | "tag_name": "local_cluster", | "fixed_value": "itau-service-provider" | }, | { | "tag_name": "consul.source.service", | "fixed_value": "itau-service-provider" | }, | { | "tag_name": "consul.source.namespace", | "fixed_value": "default" | }, | { | "tag_name": "consul.source.partition", | "fixed_value": "default" | }, | { | "tag_name": "consul.source.datacenter", | "fixed_value": "dc-gestao-comercial" | } | ], | "use_all_default_tags": true | }, | "tracing": { | "http": { | "name": "envoy.tracers.zipkin", | "typedConfig": { | "@type": "type.googleapis.com/envoy.config.trace.v3.ZipkinConfig", | "collector_cluster": "zipkin", | "collector_endpoint_version": "HTTP_JSON", | "collector_endpoint": "/api/v2/spans", | "shared_span_context": false | } | } | }, | "dynamic_resources": { | "lds_config": { | "ads": {}, | "resource_api_version": "V3" | }, | "cds_config": { | "ads": {}, | "resource_api_version": "V3" | }, | "ads_config": { | "api_type": "DELTA_GRPC", | "transport_api_version": "V3", | "grpc_services": { | "initial_metadata": [ | { | "key": "x-consul-token", | "value": "" | } | ], | "envoy_grpc": { | "cluster_name": "consul-dataplane" | } | } | } | } | } 2023-01-23T22:35:24.226Z [INFO] consul-dataplane: dns proxy disabled: configure the Consul DNS port to enable 2023-01-23T22:35:24.226Z [DEBUG] consul-dataplane: running envoy proxy: command="/usr/local/bin/envoy --config-path /consul/connect-inject/envoy-173d10ff162a0164-bootstrap.json --log-format %Y-%m-%dT%T.%eZ%z [%l] envoy.%n(%t) %v --log-level debug --disable-hot-restart --concurrency 2" 2023-01-23T22:35:24.227Z [INFO] consul-dataplane.metrics: starting merged metrics server: address=127.0.0.1:20100 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) initializing epoch 0 (base id=0, hot restart version=disabled) 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) statically linked extensions: 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.udp_proxy 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.resource_monitors: envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.retry_priorities: envoy.retry_priorities.previous_priorities 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.bootstrap: envoy.bootstrap.internal_listener, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.dubbo_proxy.filters: envoy.filters.dubbo.router 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.upstream_options: envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.upstreams.http.http_protocol_options 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.header 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.http.cache: envoy.extensions.http.cache.simple 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.filters.http.upstream: envoy.buffer, envoy.filters.http.admission_control, envoy.filters.http.buffer, envoy.filters.http.upstream_codec 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.access_loggers: envoy.access_loggers.file, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.formatter: envoy.formatter.metadata, envoy.formatter.req_without_query 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) network.connection.client: default, envoy_internal 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.matching.http.input: envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.trie_matcher 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.route.early_data_policy: envoy.route.early_data_policy.default 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.common.key_value: envoy.key_value.file_based 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.tracers: envoy.dynamic.ot, envoy.tracers.datadog, envoy.tracers.dynamic_ot, envoy.tracers.opencensus, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.path.match: envoy.path.match.uri_template.uri_template_matcher 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.filters.listener: envoy.filters.listener.http_inspector, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.thrift_proxy.transports: auto, framed, header, unframed 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.regex_engines: envoy.regex_engines.google_re2 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.request_id: envoy.request_id.uuid 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.upstreams: envoy.filters.connection_pools.tcp.generic 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.health_checkers: envoy.health_checkers.redis, envoy.health_checkers.thrift 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.matching.action: composite-action, skip 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.resolvers: envoy.ip 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.http.header_validators: envoy.http.header_validators.envoy_default 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls 2023-01-23T22:35:24.265Z+00:00 [info] envoy.main(16) envoy.matching.input_matchers: envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.ip 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.dubbo_proxy.protocols: dubbo 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.grpc_credentials: envoy.grpc_credentials.aws_iam, envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.dubbo_proxy.serializers: dubbo.hessian2 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.fault, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.cors, envoy.filters.http.csrf, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_system_buffer, envoy.filters.http.gcp_authn, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.wasm, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.quic.proof_source: envoy.quic.proof_source.filter_chain 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.trie_matcher 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.clusters: envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.dynamic_forward_proxy, envoy.clusters.redis 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.spiffe 2023-01-23T22:35:24.266Z+00:00 [info] envoy.main(16) envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo 2023-01-23T22:35:24.270Z+00:00 [warning] envoy.misc(16) Deprecated field: type envoy.config.cluster.v3.Cluster Using deprecated option 'envoy.config.cluster.v3.Cluster.http2_protocol_options' from file cluster.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override. 2023-01-23T22:35:24.270Z+00:00 [warning] envoy.misc(16) Deprecated field: type envoy.config.cluster.v3.Cluster Using deprecated option 'envoy.config.cluster.v3.Cluster.http_protocol_options' from file cluster.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override. 2023-01-23T22:35:24.270Z+00:00 [warning] envoy.misc(16) Deprecated field: type envoy.config.bootstrap.v3.Bootstrap Using deprecated option 'envoy.config.bootstrap.v3.Bootstrap.tracing' from file bootstrap.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override. 2023-01-23T22:35:24.270Z+00:00 [warning] envoy.misc(16) Deprecated field: type envoy.config.bootstrap.v3.Admin Using deprecated option 'envoy.config.bootstrap.v3.Admin.access_log_path' from file bootstrap.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override. 2023-01-23T22:35:24.270Z+00:00 [info] envoy.main(16) HTTP header map info: 2023-01-23T22:35:24.272Z+00:00 [info] envoy.main(16) request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-max-retries,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-proto,x-ot-span-context,x-request-id 2023-01-23T22:35:24.272Z+00:00 [info] envoy.main(16) request trailer map: 128 bytes: 2023-01-23T22:35:24.272Z+00:00 [info] envoy.main(16) response header map: 440 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id 2023-01-23T22:35:24.272Z+00:00 [info] envoy.main(16) response trailer map: 152 bytes: grpc-message,grpc-status 2023-01-23T22:35:24.281Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.shrink_heap. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.reduce_timeouts. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.stop_accepting_connections. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.reject_incoming_connections. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.reset_high_memory_stream. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.reduce_timeouts. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.stop_accepting_connections. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.reject_incoming_connections. 2023-01-23T22:35:24.282Z+00:00 [debug] envoy.main(16) No overload action is configured for envoy.overload_actions.reset_high_memory_stream. 2023-01-23T22:35:24.282Z+00:00 [info] envoy.main(16) runtime: layers: - name: base static_layer: re2.max_program_size.error_level: 1048576 2023-01-23T22:35:24.283Z+00:00 [info] envoy.admin(16) admin address: 127.0.0.1:19000 2023-01-23T22:35:24.284Z+00:00 [debug] envoy.misc(16) create DNS resolver type: envoy.network.dns_resolver.cares 2023-01-23T22:35:24.284Z+00:00 [debug] envoy.dns(16) c-ares library initialized. 2023-01-23T22:35:24.284Z+00:00 [info] envoy.config(16) loading tracing configuration 2023-01-23T22:35:24.284Z+00:00 [info] envoy.config(16) validating default server-wide tracing driver: envoy.tracers.zipkin 2023-01-23T22:35:24.284Z+00:00 [info] envoy.config(16) loading 0 static secret(s) 2023-01-23T22:35:24.284Z+00:00 [info] envoy.config(16) loading 3 cluster(s) 2023-01-23T22:35:24.285Z+00:00 [debug] envoy.grpc(20) completionThread running 2023-01-23T22:35:24.314Z+00:00 [debug] envoy.config(16) upstream http filter #0 2023-01-23T22:35:24.314Z+00:00 [debug] envoy.config(16) name: envoy.filters.http.upstream_codec 2023-01-23T22:35:24.314Z+00:00 [debug] envoy.config(16) config: {"@type":"type.googleapis.com/envoy.extensions.filters.http.upstream_codec.v3.UpstreamCodec"} 2023-01-23T22:35:24.314Z+00:00 [debug] envoy.upstream(16) transport socket match, socket default selected for host with address 127.0.0.1:46559 2023-01-23T22:35:24.337Z+00:00 [debug] envoy.config(16) upstream http filter #0 2023-01-23T22:35:24.337Z+00:00 [debug] envoy.config(16) name: envoy.filters.http.upstream_codec 2023-01-23T22:35:24.337Z+00:00 [debug] envoy.config(16) config: {"@type":"type.googleapis.com/envoy.extensions.filters.http.upstream_codec.v3.UpstreamCodec"} 2023-01-23T22:35:24.368Z+00:00 [debug] envoy.config(16) upstream http filter #0 2023-01-23T22:35:24.368Z+00:00 [debug] envoy.config(16) name: envoy.filters.http.upstream_codec 2023-01-23T22:35:24.368Z+00:00 [debug] envoy.config(16) config: {"@type":"type.googleapis.com/envoy.extensions.filters.http.upstream_codec.v3.UpstreamCodec"} 2023-01-23T22:35:24.368Z+00:00 [debug] envoy.upstream(16) transport socket match, socket default selected for host with address 127.0.0.1:20100 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) initializing Primary cluster consul-dataplane completed 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.init(16) init manager Cluster consul-dataplane contains no targets 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.init(16) init manager Cluster consul-dataplane initialized, notifying ClusterImplBase 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) adding TLS cluster consul-dataplane 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) membership update for TLS cluster consul-dataplane added 1 removed 0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) cm init: init complete: cluster=consul-dataplane primary=0 secondary=0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) maybe finish initialize state: 0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) cm init: adding: cluster=consul-dataplane primary=0 secondary=0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) initializing Primary cluster prometheus_backend completed 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.init(16) init manager Cluster prometheus_backend contains no targets 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.init(16) init manager Cluster prometheus_backend initialized, notifying ClusterImplBase 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) adding TLS cluster prometheus_backend 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) membership update for TLS cluster prometheus_backend added 1 removed 0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) cm init: init complete: cluster=prometheus_backend primary=0 secondary=0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) maybe finish initialize state: 0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) cm init: adding: cluster=prometheus_backend primary=0 secondary=0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.dns(16) dns resolution for jaeger-collector.observability-system started 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) cm init: adding: cluster=zipkin primary=1 secondary=0 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) maybe finish initialize state: 1 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.upstream(16) maybe finish initialize primary init clusters empty: false 2023-01-23T22:35:24.369Z+00:00 [debug] envoy.config(16) Establishing new gRPC bidi stream to consul-dataplane for rpc DeltaAggregatedResources(stream .envoy.service.discovery.v3.DeltaDiscoveryRequest) returns (stream .envoy.service.discovery.v3.DeltaDiscoveryResponse); 2023-01-23T22:35:24.370Z+00:00 [debug] envoy.router(16) [C0][S15886727338980432548] cluster 'consul-dataplane' match for URL '/envoy.service.discovery.v3.AggregatedDiscoveryService/DeltaAggregatedResources' 2023-01-23T22:35:24.370Z+00:00 [debug] envoy.router(16) [C0][S15886727338980432548] router decoding headers: ':method', 'POST' ':path', '/envoy.service.discovery.v3.AggregatedDiscoveryService/DeltaAggregatedResources' ':authority', 'consul-dataplane' ':scheme', 'http' 'te', 'trailers' 'content-type', 'application/grpc' 'x-envoy-internal', 'true' 'x-forwarded-for', 'MASK_IP.137.52' 2023-01-23T22:35:24.370Z+00:00 [debug] envoy.pool(16) queueing stream due to no available connections (ready=0 busy=0 connecting=0) 2023-01-23T22:35:24.370Z+00:00 [debug] envoy.pool(16) trying to create new connection 2023-01-23T22:35:24.370Z+00:00 [debug] envoy.pool(16) creating a new connection (connecting=0) 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.http2(16) [C0] updating connection-level initial window size to 268435456 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.connection(16) [C0] current connecting state: true 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.client(16) [C0] connecting 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.connection(16) [C0] connecting to 127.0.0.1:46559 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.connection(16) [C0] connection in progress 2023-01-23T22:35:24.377Z+00:00 [info] envoy.config(16) loading 1 listener(s) 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.config(16) listener #0: 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.config(16) begin add/update listener: name=envoy_prometheus_metrics_listener hash=9133596812378588937 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.config(16) use full listener update path for listener name=envoy_prometheus_metrics_listener hash=9133596812378588937 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.config(16) filter #0: 2023-01-23T22:35:24.377Z+00:00 [debug] envoy.config(16) name: envoy.filters.network.http_connection_manager 2023-01-23T22:35:24.378Z+00:00 [debug] envoy.config(16) config: {"@type":"type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager","stat_prefix":"envoy_prometheus_metrics","codec_type":"HTTP1","route_config":{"name":"self_admin_route","virtual_hosts":[{"name":"self_admin","domains":["*"],"routes":[{"match":{"path":"/metrics"},"route":{"cluster":"prometheus_backend","prefix_rewrite":"/stats/prometheus"}},{"match":{"prefix":"/"},"direct_response":{"status":404}}]}]},"http_filters":[{"name":"envoy.filters.http.router","typed_config":{"@type":"type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"}}]} 2023-01-23T22:35:24.382Z+00:00 [debug] envoy.config(16) http filter #0 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) name: envoy.filters.http.router 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) config: {"@type":"type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"} 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) new fc_contexts has 1 filter chains, including 1 newly built 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.init(16) added target Listener-init-target envoy_prometheus_metrics_listener to init manager Server 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) Create listen socket for listener envoy_prometheus_metrics_listener on address 0.0.0.0:20200 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) envoy_prometheus_metrics_listener: Setting socket options succeeded 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) Set listener envoy_prometheus_metrics_listener socket factory local address to 0.0.0.0:20200 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) Create listen socket for listener envoy_prometheus_metrics_listener on address 0.0.0.0:20200 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) envoy_prometheus_metrics_listener: Setting socket options succeeded 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.config(16) add active listener: name=envoy_prometheus_metrics_listener, hash=9133596812378588937, tag=1, address=0.0.0.0:20200 2023-01-23T22:35:24.384Z+00:00 [info] envoy.config(16) loading stats configuration 2023-01-23T22:35:24.384Z+00:00 [debug] envoy.init(16) added target LDS to init manager Server 2023-01-23T22:35:24.385Z+00:00 [info] envoy.main(16) starting main dispatch loop 2023-01-23T22:35:24.385Z+00:00 [debug] envoy.connection(16) [C0] connected 2023-01-23T22:35:24.385Z+00:00 [debug] envoy.client(16) [C0] connected 2023-01-23T22:35:24.385Z+00:00 [debug] envoy.pool(16) [C0] attaching to next stream 2023-01-23T22:35:24.385Z+00:00 [debug] envoy.pool(16) [C0] creating stream 2023-01-23T22:35:24.385Z+00:00 [debug] envoy.router(16) [C0][S15886727338980432548] pool ready 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.dns(16) dns resolution for jaeger-collector.observability-system completed with status 0 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) transport socket match, socket default selected for host with address MASK_IP.183.119:9411 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) DNS hosts have changed for jaeger-collector.observability-system 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) DNS refresh rate reset for jaeger-collector.observability-system, refresh rate 5000 ms 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) initializing Primary cluster zipkin completed 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.init(16) init manager Cluster zipkin contains no targets 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.init(16) init manager Cluster zipkin initialized, notifying ClusterImplBase 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) adding TLS cluster zipkin 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) membership update for TLS cluster zipkin added 1 removed 0 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) cm init: init complete: cluster=zipkin primary=0 secondary=0 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) maybe finish initialize state: 1 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) maybe finish initialize primary init clusters empty: true 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.init(16) init manager RTDS contains no targets 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.init(16) init manager RTDS initialized, notifying RTDS 2023-01-23T22:35:24.390Z+00:00 [info] envoy.runtime(16) RTDS has finished initialization 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) continue initializing secondary clusters 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) maybe finish initialize state: 2 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) maybe finish initialize primary init clusters empty: true 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) maybe finish initialize secondary init clusters empty: true 2023-01-23T22:35:24.390Z+00:00 [debug] envoy.upstream(16) maybe finish initialize cds api ready: true 2023-01-23T22:35:24.390Z+00:00 [info] envoy.upstream(16) cm init: initializing cds 2023-01-23T22:35:24.390Z+00:00 [warning] envoy.main(16) there is no configured limit to the number of allowed active connections. Set a limit via the runtime key overload.global_downstream_max_connections 2023-01-23T22:35:24.595Z+00:00 [debug] envoy.router(16) [C0][S15886727338980432548] upstream headers complete: end_stream=false 2023-01-23T22:35:24.597Z+00:00 [debug] envoy.http(16) async http request response headers (end_stream=false): ':status', '200' 'content-type', 'application/grpc' 2023-01-23T22:35:24.597Z+00:00 [debug] envoy.config(16) Received DeltaDiscoveryResponse for type.googleapis.com/envoy.config.cluster.v3.Cluster at version 2023-01-23T22:35:24.598Z+00:00 [info] envoy.upstream(16) cds: add 5 cluster(s), remove 0 cluster(s) ```

Environment details

lkysow commented 1 year ago

@nascimento how did you resolve the issue (for future readers)?