search

hashicorp / consul

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.
https://www.consul.io
Other
28.37k stars 4.42k forks source link

Unable to get Consul Connect Envoy working with TLS Enabled #7023

Closed allthingsclowd closed 4 years ago

allthingsclowd commented 4 years ago

Overview of the Issue

I am trying to setup a very basic demonstration of Consul Connect Envoy when TLS is enabled but I cannot get it to work. The internal proxy works fine with TLS enabled.

Please note I have very limited Envoy experience but after following the documentation I keep getting this Error with the Envoy Proxy

Unable to parse JSON as proto (INVALID_ARGUMENT:(static_resources.clusters[0].tls_context.common_tls_context.validation_context.trusted_ca.inline_bytes): invalid value "[45 45 45 45 45 6

Reproduction Steps

Simple Demo of How to Setup an Envoy Connect Service when Consul is secured with TLS

The goal of this test was to be able to demonstrate a productionised version of a Consul Connect Envoy Service. All online examples today rely on Docker and no TLS - many customers still don't allow Docker in production (I know, unbelieveable!). So I was attempting to running the Envoy proxy directly on Ubuntu without Docker.

Deployment Platforms Tested

or

Consul

It's a 3 host setup with a single node Consul Server and two separate Consul Agent nodes running the envoy proxies. I have used all the default settings for Consul with the following exceptions:

HCL Config Server

datacenter = "allthingscloud1"
data_dir = "/usr/local/consul"
encrypt = "mUIJq6TITeenfVa2yMSi6yLwxrz2AYcC0dXissYpOxE="
log_level = "INFO"
server = true
node_name = "leader010"
addresses {
    https = "0.0.0.0"
}
ports {
    https = 8321
    http = -1
    grpc = 8502
}
connect {
    enabled = true
}
verify_incoming = true
verify_outgoing = true
key_file = "/etc/consul.d/pki/tls/private/consul/server-key.pem"
cert_file = "/etc/consul.d/pki/tls/certs/consul/server.pem"

App Server HCL

vagrant@app01:~$ cat /etc/consul.d/consul.hcl
datacenter = "allthingscloud1"
data_dir = "/usr/local/consul"
encrypt = "mUIJq6TITeenfVa2yMSi6yLwxrz2AYcC0dXissYpOxE="
log_level = "INFO"
server = true
node_name = "leader010"
addresses {
    https = "0.0.0.0"
}
ports {
    https = 8321
    http = -1
    grpc = 8502
}
connect {
    enabled = true
}
verify_incoming = true
verify_outgoing = true
key_file = "/etc/consul.d/pki/tls/private/consul/server-key.pem"
cert_file = "/etc/consul.d/pki/tls/certs/consul/server.pem"
ca_file = "/etc/consul.d/pki/tls/certs/consul/consul-ca.pem"

Set up environment variables

# Configure consul environment variables for use with certificates 
export CONSUL_HTTP_ADDR=https://127.0.0.1:8321
export CONSUL_CACERT=/usr/local/bootstrap/certificate-config/consul-ca.pem
export CONSUL_CLIENT_CERT=/usr/local/bootstrap/certificate-config/cli.pem
export CONSUL_CLIENT_KEY=/usr/local/bootstrap/certificate-config/cli-key.pem

export VAULT_TOKEN=reallystrongpassword
export VAULT_ADDR=https://192.168.4.11:8322
export VAULT_CLIENT_KEY=/usr/local/bootstrap/certificate-config/client-key.pem
export VAULT_CLIENT_CERT=/usr/local/bootstrap/certificate-config/client.pem
export VAULT_CACERT=/usr/local/bootstrap/certificate-config/consul-ca.pem

AGENTTOKEN=`vault kv get -field "value" kv/development/consulagentacl`
export CONSUL_HTTP_TOKEN=${AGENTTOKEN}
export CONSUL_HTTP_SSL=true
export CONSUL_GRPC_ADDR=127.0.0.1:8502

SERVICETOKEN=`vault kv get -field "value" kv/development/SERVICETOKEN`
export SERVICETOKEN

Everything else is left at Consul Defaults.

Command used to launch envoy proxy

vagrant@app01:~$ /usr/local/bin/consul connect envoy \
                          -http-addr=https://127.0.0.1:8321 \
                          -ca-file=/usr/local/bootstrap/certificate-config/consul-ca.pem \
                          -client-cert=/usr/local/bootstrap/certificate-config/cli.pem \
                          -client-key=/usr/local/bootstrap/certificate-config/cli-key.pem \
                          -token=${SERVICETOKEN} \
                          -sidecar-for httpecho -- -l debug &

Result

vagrant@app01:~$ /usr/local/bin/consul connect envoy -http-addr=https://127.0.0.1:8321 -ca-file=/usr/local/bootstrap/certificate-config/consul-ca.pem -client-cert=/usr/local/bootstrap/certificate-config/cli.pem -client-key=/usr/local/bootstrap/certificate-config/cli-key.pem -token=${SERVICETOKEN} -sidecar-for httpecho -- -l debug &
[1] 1427
vagrant@app01:~$ [2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:238] initializing epoch 0 (hot restart version=disabled)
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:240] statically linked extensions:
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:242]   access_loggers: envoy.file_access_log,envoy.http_grpc_access_log
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:245]   filters.http: envoy.buffer,envoy.cors,envoy.csrf,envoy.ext_authz,envoy.fault,envoy.filters.http.dynamic_forward_proxy,envoy.filters.http.grpc_http1_reverse_bridge,envoy.filters.http.header_to_metadata,envoy.filters.http.jwt_authn,envoy.filters.http.original_src,envoy.filters.http.rbac,envoy.filters.http.tap,envoy.grpc_http1_bridge,envoy.grpc_json_transcoder,envoy.grpc_web,envoy.gzip,envoy.health_check,envoy.http_dynamo_filter,envoy.ip_tagging,envoy.lua,envoy.rate_limit,envoy.router,envoy.squash
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:248]   filters.listener: envoy.listener.original_dst,envoy.listener.original_src,envoy.listener.proxy_protocol,envoy.listener.tls_inspector
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:251]   filters.network: envoy.client_ssl_auth,envoy.echo,envoy.ext_authz,envoy.filters.network.dubbo_proxy,envoy.filters.network.mysql_proxy,envoy.filters.network.rbac,envoy.filters.network.sni_cluster,envoy.filters.network.thrift_proxy,envoy.filters.network.zookeeper_proxy,envoy.http_connection_manager,envoy.mongo_proxy,envoy.ratelimit,envoy.redis_proxy,envoy.tcp_proxy
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:253]   stat_sinks: envoy.dog_statsd,envoy.metrics_service,envoy.stat_sinks.hystrix,envoy.statsd
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:255]   tracers: envoy.dynamic.ot,envoy.lightstep,envoy.tracers.datadog,envoy.tracers.opencensus,envoy.zipkin
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:258]   transport_sockets.downstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:261]   transport_sockets.upstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls
[2020-01-09 22:38:37.024][1427][info][main] [external/envoy/source/server/server.cc:267] buffer implementation: old (libevent)
[2020-01-09 22:38:37.030][1427][critical][main] [external/envoy/source/server/server.cc:93] error initializing configuration '/tmp/envoy-15e858fdea3d43c6-bootstrap.json': Unable to parse JSON as proto (INVALID_ARGUMENT:(static_resources.clusters[0].tls_context.common_tls_context.validation_context.trusted_ca.inline_bytes): invalid value "[45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 67 55 84 67 67 65 112 83 103 65 119 73 66 65 103 73 82 65 77 51 114 82 68 116 43 117 69 103 72 76 99 48 97 86 87 75 74 118 78 77 119 67 103 89 73 75 111 90 73 122 106 48 69 65 119 73 119 103 98 107 120 10 67 122 65 74 66 103 78 86 66 65 89 84 65 108 86 84 77 81 115 119 67 81 89 68 86 81 81 73 69 119 74 68 81 84 69 87 77 66 81 71 65 49 85 69 66 120 77 78 85 50 70 117 73 69 90 121 89 87 53 106 97 88 78 106 10 98 122 69 97 77 66 103 71 65 49 85 69 67 82 77 82 77 84 65 120 73 70 78 108 89 50 57 117 90 67 66 84 100 72 74 108 90 88 81 120 68 106 65 77 66 103 78 86 66 66 69 84 66 84 107 48 77 84 65 49 77 82 99 119 10 70 81 89 68 86 81 81 75 69 119 53 73 89 88 78 111 97 85 78 118 99 110 65 103 83 87 53 106 76 106 70 65 77 68 52 71 65 49 85 69 65 120 77 51 81 50 57 117 99 51 86 115 73 69 70 110 90 87 53 48 73 69 78 66 10 73 68 73 51 77 122 99 120 77 122 77 120 77 106 103 120 78 84 69 53 78 84 107 50 79 68 103 121 78 68 73 52 77 106 99 121 79 84 65 48 77 68 103 50 79 68 89 49 78 68 73 53 77 84 65 101 70 119 48 121 77 68 65 120 10 77 68 99 120 77 106 73 49 77 122 108 97 70 119 48 121 78 84 65 120 77 68 85 120 77 106 73 49 77 122 108 97 77 73 71 53 77 81 115 119 67 81 89 68 86 81 81 71 69 119 74 86 85 122 69 76 77 65 107 71 65 49 85 69 10 67 66 77 67 81 48 69 120 70 106 65 85 66 103 78 86 66 65 99 84 68 86 78 104 98 105 66 71 99 109 70 117 89 50 108 122 89 50 56 120 71 106 65 89 66 103 78 86 66 65 107 84 69 84 69 119 77 83 66 84 90 87 78 118 10 98 109 81 103 85 51 82 121 90 87 86 48 77 81 52 119 68 65 89 68 86 81 81 82 69 119 85 53 78 68 69 119 78 84 69 88 77 66 85 71 65 49 85 69 67 104 77 79 83 71 70 122 97 71 108 68 98 51 74 119 73 69 108 117 10 89 121 52 120 81 68 65 43 66 103 78 86 66 65 77 84 78 48 78 118 98 110 78 49 98 67 66 66 90 50 86 117 100 67 66 68 81 83 65 121 78 122 77 51 77 84 77 122 77 84 73 52 77 84 85 120 79 84 85 53 78 106 103 52 10 77 106 81 121 79 68 73 51 77 106 107 119 78 68 65 52 78 106 103 50 78 84 81 121 79 84 69 119 87 84 65 84 66 103 99 113 104 107 106 79 80 81 73 66 66 103 103 113 104 107 106 79 80 81 77 66 66 119 78 67 65 65 82 65 10 79 73 75 122 51 85 103 52 54 97 78 116 87 108 105 98 49 76 55 57 122 85 49 103 78 102 81 66 113 118 68 74 106 116 86 77 108 122 50 120 102 57 86 119 66 52 111 50 55 108 117 120 67 103 71 55 50 50 114 120 69 98 84 65 10 86 77 66 77 80 110 103 111 112 56 66 43 57 104 78 119 109 120 69 88 111 51 115 119 101 84 65 79 66 103 78 86 72 81 56 66 65 102 56 69 66 65 77 67 65 89 89 119 68 119 89 68 86 82 48 84 65 81 72 47 66 65 85 119 10 65 119 69 66 47 122 65 112 66 103 78 86 72 81 52 69 73 103 81 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 10 75 119 89 68 86 82 48 106 66 67 81 119 73 111 65 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 67 103 89 73 10 75 111 90 73 122 106 48 69 65 119 73 68 82 119 65 119 82 65 73 103 70 76 65 79 53 57 110 47 122 81 56 79 109 75 99 72 56 78 103 55 49 113 110 104 98 70 97 113 48 121 75 80 118 69 117 98 67 53 116 117 69 115 73 67 10 73 66 74 84 48 67 48 49 82 67 87 70 74 120 110 111 117 66 70 51 108 51 83 75 118 114 72 89 78 67 121 68 71 52 98 56 54 73 111 78 56 87 120 48 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10]" for type TYPE_BYTES):
{
  "admin": {
    "access_log_path": "/dev/null",
    "address": {
      "socket_address": {
        "address": "127.0.0.1",
        "port_value": 19000
      }
    }
  },
  "node": {
    "cluster": "httpecho",
    "id": "httpecho-sidecar-proxy"
  },
  "static_resources": {
    "clusters": [
      {
        "name": "local_agent",
        "connect_timeout": "1s",
        "type": "STATIC",
        "tls_context": {
          "common_tls_context": {
            "validation_context": {
              "trusted_ca": {
                "inline_bytes": "[45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 67 55 84 67 67 65 112 83 103 65 119 73 66 65 103 73 82 65 77 51 114 82 68 116 43 117 69 103 72 76 99 48 97 86 87 75 74 118 78 77 119 67 103 89 73 75 111 90 73 122 106 48 69 65 119 73 119 103 98 107 120 10 67 122 65 74 66 103 78 86 66 65 89 84 65 108 86 84 77 81 115 119 67 81 89 68 86 81 81 73 69 119 74 68 81 84 69 87 77 66 81 71 65 49 85 69 66 120 77 78 85 50 70 117 73 69 90 121 89 87 53 106 97 88 78 106 10 98 122 69 97 77 66 103 71 65 49 85 69 67 82 77 82 77 84 65 120 73 70 78 108 89 50 57 117 90 67 66 84 100 72 74 108 90 88 81 120 68 106 65 77 66 103 78 86 66 66 69 84 66 84 107 48 77 84 65 49 77 82 99 119 10 70 81 89 68 86 81 81 75 69 119 53 73 89 88 78 111 97 85 78 118 99 110 65 103 83 87 53 106 76 106 70 65 77 68 52 71 65 49 85 69 65 120 77 51 81 50 57 117 99 51 86 115 73 69 70 110 90 87 53 48 73 69 78 66 10 73 68 73 51 77 122 99 120 77 122 77 120 77 106 103 120 78 84 69 53 78 84 107 50 79 68 103 121 78 68 73 52 77 106 99 121 79 84 65 48 77 68 103 50 79 68 89 49 78 68 73 53 77 84 65 101 70 119 48 121 77 68 65 120 10 77 68 99 120 77 106 73 49 77 122 108 97 70 119 48 121 78 84 65 120 77 68 85 120 77 106 73 49 77 122 108 97 77 73 71 53 77 81 115 119 67 81 89 68 86 81 81 71 69 119 74 86 85 122 69 76 77 65 107 71 65 49 85 69 10 67 66 77 67 81 48 69 120 70 106 65 85 66 103 78 86 66 65 99 84 68 86 78 104 98 105 66 71 99 109 70 117 89 50 108 122 89 50 56 120 71 106 65 89 66 103 78 86 66 65 107 84 69 84 69 119 77 83 66 84 90 87 78 118 10 98 109 81 103 85 51 82 121 90 87 86 48 77 81 52 119 68 65 89 68 86 81 81 82 69 119 85 53 78 68 69 119 78 84 69 88 77 66 85 71 65 49 85 69 67 104 77 79 83 71 70 122 97 71 108 68 98 51 74 119 73 69 108 117 10 89 121 52 120 81 68 65 43 66 103 78 86 66 65 77 84 78 48 78 118 98 110 78 49 98 67 66 66 90 50 86 117 100 67 66 68 81 83 65 121 78 122 77 51 77 84 77 122 77 84 73 52 77 84 85 120 79 84 85 53 78 106 103 52 10 77 106 81 121 79 68 73 51 77 106 107 119 78 68 65 52 78 106 103 50 78 84 81 121 79 84 69 119 87 84 65 84 66 103 99 113 104 107 106 79 80 81 73 66 66 103 103 113 104 107 106 79 80 81 77 66 66 119 78 67 65 65 82 65 10 79 73 75 122 51 85 103 52 54 97 78 116 87 108 105 98 49 76 55 57 122 85 49 103 78 102 81 66 113 118 68 74 106 116 86 77 108 122 50 120 102 57 86 119 66 52 111 50 55 108 117 120 67 103 71 55 50 50 114 120 69 98 84 65 10 86 77 66 77 80 110 103 111 112 56 66 43 57 104 78 119 109 120 69 88 111 51 115 119 101 84 65 79 66 103 78 86 72 81 56 66 65 102 56 69 66 65 77 67 65 89 89 119 68 119 89 68 86 82 48 84 65 81 72 47 66 65 85 119 10 65 119 69 66 47 122 65 112 66 103 78 86 72 81 52 69 73 103 81 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 10 75 119 89 68 86 82 48 106 66 67 81 119 73 111 65 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 67 103 89 73 10 75 111 90 73 122 106 48 69 65 119 73 68 82 119 65 119 82 65 73 103 70 76 65 79 53 57 110 47 122 81 56 79 109 75 99 72 56 78 103 55 49 113 110 104 98 70 97 113 48 121 75 80 118 69 117 98 67 53 116 117 69 115 73 67 10 73 66 74 84 48 67 48 49 82 67 87 70 74 120 110 111 117 66 70 51 108 51 83 75 118 114 72 89 78 67 121 68 71 52 98 56 54 73 111 78 56 87 120 48 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10]"
              }
            }
          }
        },
        "http2_protocol_options": {},
        "hosts": [
          {
            "socket_address": {
              "address": "127.0.0.1",
              "port_value": 8502
            }
          }
        ]
      }
    ]
  },
  "stats_config": {
    "stats_tags": [
      {
        "regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.custom_hash"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.service_subset"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.service"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.namespace"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.datacenter"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.routing_type"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)",
        "tag_name": "consul.trust_domain"
      },
      {
        "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.target"
      },
      {
        "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)",
        "tag_name": "consul.full_target"
      },
      {
        "tag_name": "local_cluster",
        "fixed_value": "httpecho"
      }
    ],
    "use_all_default_tags": true
  },
  "dynamic_resources": {
    "lds_config": {
      "ads": {}
    },
    "cds_config": {
      "ads": {}
    },
    "ads_config": {
      "api_type": "GRPC",
      "grpc_services": {
        "initial_metadata": [
          {
            "key": "x-consul-token",
            "value": "f9e89413-eba4-2792-0bc4-6ce24df050aa"
          }
        ],
        "envoy_grpc": {
          "cluster_name": "local_agent"
        }
      }
    }
  }
}

[2020-01-09 22:38:37.033][1427][info][main] [external/envoy/source/server/server.cc:560] exiting
[2020-01-09 22:38:37.033][1427][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] init manager Server destroyed
Unable to parse JSON as proto (INVALID_ARGUMENT:(static_resources.clusters[0].tls_context.common_tls_context.validation_context.trusted_ca.inline_bytes): invalid value "[45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 67 55 84 67 67 65 112 83 103 65 119 73 66 65 103 73 82 65 77 51 114 82 68 116 43 117 69 103 72 76 99 48 97 86 87 75 74 118 78 77 119 67 103 89 73 75 111 90 73 122 106 48 69 65 119 73 119 103 98 107 120 10 67 122 65 74 66 103 78 86 66 65 89 84 65 108 86 84 77 81 115 119 67 81 89 68 86 81 81 73 69 119 74 68 81 84 69 87 77 66 81 71 65 49 85 69 66 120 77 78 85 50 70 117 73 69 90 121 89 87 53 106 97 88 78 106 10 98 122 69 97 77 66 103 71 65 49 85 69 67 82 77 82 77 84 65 120 73 70 78 108 89 50 57 117 90 67 66 84 100 72 74 108 90 88 81 120 68 106 65 77 66 103 78 86 66 66 69 84 66 84 107 48 77 84 65 49 77 82 99 119 10 70 81 89 68 86 81 81 75 69 119 53 73 89 88 78 111 97 85 78 118 99 110 65 103 83 87 53 106 76 106 70 65 77 68 52 71 65 49 85 69 65 120 77 51 81 50 57 117 99 51 86 115 73 69 70 110 90 87 53 48 73 69 78 66 10 73 68 73 51 77 122 99 120 77 122 77 120 77 106 103 120 78 84 69 53 78 84 107 50 79 68 103 121 78 68 73 52 77 106 99 121 79 84 65 48 77 68 103 50 79 68 89 49 78 68 73 53 77 84 65 101 70 119 48 121 77 68 65 120 10 77 68 99 120 77 106 73 49 77 122 108 97 70 119 48 121 78 84 65 120 77 68 85 120 77 106 73 49 77 122 108 97 77 73 71 53 77 81 115 119 67 81 89 68 86 81 81 71 69 119 74 86 85 122 69 76 77 65 107 71 65 49 85 69 10 67 66 77 67 81 48 69 120 70 106 65 85 66 103 78 86 66 65 99 84 68 86 78 104 98 105 66 71 99 109 70 117 89 50 108 122 89 50 56 120 71 106 65 89 66 103 78 86 66 65 107 84 69 84 69 119 77 83 66 84 90 87 78 118 10 98 109 81 103 85 51 82 121 90 87 86 48 77 81 52 119 68 65 89 68 86 81 81 82 69 119 85 53 78 68 69 119 78 84 69 88 77 66 85 71 65 49 85 69 67 104 77 79 83 71 70 122 97 71 108 68 98 51 74 119 73 69 108 117 10 89 121 52 120 81 68 65 43 66 103 78 86 66 65 77 84 78 48 78 118 98 110 78 49 98 67 66 66 90 50 86 117 100 67 66 68 81 83 65 121 78 122 77 51 77 84 77 122 77 84 73 52 77 84 85 120 79 84 85 53 78 106 103 52 10 77 106 81 121 79 68 73 51 77 106 107 119 78 68 65 52 78 106 103 50 78 84 81 121 79 84 69 119 87 84 65 84 66 103 99 113 104 107 106 79 80 81 73 66 66 103 103 113 104 107 106 79 80 81 77 66 66 119 78 67 65 65 82 65 10 79 73 75 122 51 85 103 52 54 97 78 116 87 108 105 98 49 76 55 57 122 85 49 103 78 102 81 66 113 118 68 74 106 116 86 77 108 122 50 120 102 57 86 119 66 52 111 50 55 108 117 120 67 103 71 55 50 50 114 120 69 98 84 65 10 86 77 66 77 80 110 103 111 112 56 66 43 57 104 78 119 109 120 69 88 111 51 115 119 101 84 65 79 66 103 78 86 72 81 56 66 65 102 56 69 66 65 77 67 65 89 89 119 68 119 89 68 86 82 48 84 65 81 72 47 66 65 85 119 10 65 119 69 66 47 122 65 112 66 103 78 86 72 81 52 69 73 103 81 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 10 75 119 89 68 86 82 48 106 66 67 81 119 73 111 65 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 67 103 89 73 10 75 111 90 73 122 106 48 69 65 119 73 68 82 119 65 119 82 65 73 103 70 76 65 79 53 57 110 47 122 81 56 79 109 75 99 72 56 78 103 55 49 113 110 104 98 70 97 113 48 121 75 80 118 69 117 98 67 53 116 117 69 115 73 67 10 73 66 74 84 48 67 48 49 82 67 87 70 74 120 110 111 117 66 70 51 108 51 83 75 118 114 72 89 78 67 121 68 71 52 98 56 54 73 111 78 56 87 120 48 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10]" for type TYPE_BYTES): {
  "admin": {
    "access_log_path": "/dev/null",
    "address": {
      "socket_address": {
        "address": "127.0.0.1",
        "port_value": 19000
      }
    }
  },
  "node": {
    "cluster": "httpecho",
    "id": "httpecho-sidecar-proxy"
  },
  "static_resources": {
    "clusters": [
      {
        "name": "local_agent",
        "connect_timeout": "1s",
        "type": "STATIC",
        "tls_context": {
          "common_tls_context": {
            "validation_context": {
              "trusted_ca": {
                "inline_bytes": "[45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 67 55 84 67 67 65 112 83 103 65 119 73 66 65 103 73 82 65 77 51 114 82 68 116 43 117 69 103 72 76 99 48 97 86 87 75 74 118 78 77 119 67 103 89 73 75 111 90 73 122 106 48 69 65 119 73 119 103 98 107 120 10 67 122 65 74 66 103 78 86 66 65 89 84 65 108 86 84 77 81 115 119 67 81 89 68 86 81 81 73 69 119 74 68 81 84 69 87 77 66 81 71 65 49 85 69 66 120 77 78 85 50 70 117 73 69 90 121 89 87 53 106 97 88 78 106 10 98 122 69 97 77 66 103 71 65 49 85 69 67 82 77 82 77 84 65 120 73 70 78 108 89 50 57 117 90 67 66 84 100 72 74 108 90 88 81 120 68 106 65 77 66 103 78 86 66 66 69 84 66 84 107 48 77 84 65 49 77 82 99 119 10 70 81 89 68 86 81 81 75 69 119 53 73 89 88 78 111 97 85 78 118 99 110 65 103 83 87 53 106 76 106 70 65 77 68 52 71 65 49 85 69 65 120 77 51 81 50 57 117 99 51 86 115 73 69 70 110 90 87 53 48 73 69 78 66 10 73 68 73 51 77 122 99 120 77 122 77 120 77 106 103 120 78 84 69 53 78 84 107 50 79 68 103 121 78 68 73 52 77 106 99 121 79 84 65 48 77 68 103 50 79 68 89 49 78 68 73 53 77 84 65 101 70 119 48 121 77 68 65 120 10 77 68 99 120 77 106 73 49 77 122 108 97 70 119 48 121 78 84 65 120 77 68 85 120 77 106 73 49 77 122 108 97 77 73 71 53 77 81 115 119 67 81 89 68 86 81 81 71 69 119 74 86 85 122 69 76 77 65 107 71 65 49 85 69 10 67 66 77 67 81 48 69 120 70 106 65 85 66 103 78 86 66 65 99 84 68 86 78 104 98 105 66 71 99 109 70 117 89 50 108 122 89 50 56 120 71 106 65 89 66 103 78 86 66 65 107 84 69 84 69 119 77 83 66 84 90 87 78 118 10 98 109 81 103 85 51 82 121 90 87 86 48 77 81 52 119 68 65 89 68 86 81 81 82 69 119 85 53 78 68 69 119 78 84 69 88 77 66 85 71 65 49 85 69 67 104 77 79 83 71 70 122 97 71 108 68 98 51 74 119 73 69 108 117 10 89 121 52 120 81 68 65 43 66 103 78 86 66 65 77 84 78 48 78 118 98 110 78 49 98 67 66 66 90 50 86 117 100 67 66 68 81 83 65 121 78 122 77 51 77 84 77 122 77 84 73 52 77 84 85 120 79 84 85 53 78 106 103 52 10 77 106 81 121 79 68 73 51 77 106 107 119 78 68 65 52 78 106 103 50 78 84 81 121 79 84 69 119 87 84 65 84 66 103 99 113 104 107 106 79 80 81 73 66 66 103 103 113 104 107 106 79 80 81 77 66 66 119 78 67 65 65 82 65 10 79 73 75 122 51 85 103 52 54 97 78 116 87 108 105 98 49 76 55 57 122 85 49 103 78 102 81 66 113 118 68 74 106 116 86 77 108 122 50 120 102 57 86 119 66 52 111 50 55 108 117 120 67 103 71 55 50 50 114 120 69 98 84 65 10 86 77 66 77 80 110 103 111 112 56 66 43 57 104 78 119 109 120 69 88 111 51 115 119 101 84 65 79 66 103 78 86 72 81 56 66 65 102 56 69 66 65 77 67 65 89 89 119 68 119 89 68 86 82 48 84 65 81 72 47 66 65 85 119 10 65 119 69 66 47 122 65 112 66 103 78 86 72 81 52 69 73 103 81 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 10 75 119 89 68 86 82 48 106 66 67 81 119 73 111 65 103 83 117 75 111 105 86 47 47 51 76 100 122 51 66 57 117 81 54 99 68 113 72 47 77 66 83 114 99 86 72 83 87 69 103 110 43 108 110 120 52 47 73 77 119 67 103 89 73 10 75 111 90 73 122 106 48 69 65 119 73 68 82 119 65 119 82 65 73 103 70 76 65 79 53 57 110 47 122 81 56 79 109 75 99 72 56 78 103 55 49 113 110 104 98 70 97 113 48 121 75 80 118 69 117 98 67 53 116 117 69 115 73 67 10 73 66 74 84 48 67 48 49 82 67 87 70 74 120 110 111 117 66 70 51 108 51 83 75 118 114 72 89 78 67 121 68 71 52 98 56 54 73 111 78 56 87 120 48 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10]"
              }
            }
          }
        },
        "http2_protocol_options": {},
        "hosts": [
          {
            "socket_address": {
              "address": "127.0.0.1",
              "port_value": 8502
            }
          }
        ]
      }
    ]
  },
  "stats_config": {
    "stats_tags": [
      {
        "regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.custom_hash"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.service_subset"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.service"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.namespace"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.datacenter"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.routing_type"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)",
        "tag_name": "consul.trust_domain"
      },
      {
        "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.target"
      },
      {
        "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)",
        "tag_name": "consul.full_target"
      },
      {
        "tag_name": "local_cluster",
        "fixed_value": "httpecho"
      }
    ],
    "use_all_default_tags": true
  },
  "dynamic_resources": {
    "lds_config": {
      "ads": {}
    },
    "cds_config": {
      "ads": {}
    },
    "ads_config": {
      "api_type": "GRPC",
      "grpc_services": {
        "initial_metadata": [
          {
            "key": "x-consul-token",
            "value": "f9e89413-eba4-2792-0bc4-6ce24df050aa"
          }
        ],
        "envoy_grpc": {
          "cluster_name": "local_agent"
        }
      }
    }
  }
}
allthingsclowd commented 4 years ago

Resolved with the above PR #7024 - thank you

vagrant@app01:~$ /usr/local/bin/consul connect envoy -http-addr=https://127.0.0.1:8321 -ca-file=/usr/local/bootstrap/certificate-config/consul-ca.pem -client-cert=/usr/local/bootstrap/certificate-config/cli.pem -client-key=/usr/local/bootstrap/certificate-config/cli-key.pem -token=${SERVICETOKEN} -sidecar-for httpecho -bootstrap
{
  "admin": {
    "access_log_path": "/dev/null",
    "address": {
      "socket_address": {
        "address": "127.0.0.1",
        "port_value": 19000
      }
    }
  },
  "node": {
    "cluster": "httpecho",
    "id": "httpecho-sidecar-proxy"
  },
  "static_resources": {
    "clusters": [
      {
        "name": "local_agent",
        "connect_timeout": "1s",
        "type": "STATIC",
        "tls_context": {
          "common_tls_context": {
            "validation_context": {
              "trusted_ca": {
                "inline_string": "-----BEGIN CERTIFICATE-----\nMIIC7TCCApSgAwIBAgIRAM3rRDt+uEgHLc0aVWKJvNMwCgYIKoZIzj0EAwIwgbkx\nCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJDQTEWMBQGA1UEBxMNU2FuIEZyYW5jaXNj\nbzEaMBgGA1UECRMRMTAxIFNlY29uZCBTdHJlZXQxDjAMBgNVBBETBTk0MTA1MRcw\nFQYDVQQKEw5IYXNoaUNvcnAgSW5jLjFAMD4GA1UEAxM3Q29uc3VsIEFnZW50IENB\nIDI3MzcxMzMxMjgxNTE5NTk2ODgyNDI4MjcyOTA0MDg2ODY1NDI5MTAeFw0yMDAx\nMDcxMjI1MzlaFw0yNTAxMDUxMjI1MzlaMIG5MQswCQYDVQQGEwJVUzELMAkGA1UE\nCBMCQ0ExFjAUBgNVBAcTDVNhbiBGcmFuY2lzY28xGjAYBgNVBAkTETEwMSBTZWNv\nbmQgU3RyZWV0MQ4wDAYDVQQREwU5NDEwNTEXMBUGA1UEChMOSGFzaGlDb3JwIElu\nYy4xQDA+BgNVBAMTN0NvbnN1bCBBZ2VudCBDQSAyNzM3MTMzMTI4MTUxOTU5Njg4\nMjQyODI3MjkwNDA4Njg2NTQyOTEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARA\nOIKz3Ug46aNtWlib1L79zU1gNfQBqvDJjtVMlz2xf9VwB4o27luxCgG722rxEbTA\nVMBMPngop8B+9hNwmxEXo3sweTAOBgNVHQ8BAf8EBAMCAYYwDwYDVR0TAQH/BAUw\nAwEB/zApBgNVHQ4EIgQgSuKoiV//3Ldz3B9uQ6cDqH/MBSrcVHSWEgn+lnx4/IMw\nKwYDVR0jBCQwIoAgSuKoiV//3Ldz3B9uQ6cDqH/MBSrcVHSWEgn+lnx4/IMwCgYI\nKoZIzj0EAwIDRwAwRAIgFLAO59n/zQ8OmKcH8Ng71qnhbFaq0yKPvEubC5tuEsIC\nIBJT0C01RCWFJxnouBF3l3SKvrHYNCyDG4b86IoN8Wx0\n-----END CERTIFICATE-----\n"
              }
            }
          }
        },
        "http2_protocol_options": {},
        "hosts": [
          {
            "socket_address": {
              "address": "127.0.0.1",
              "port_value": 8502
            }
          }
        ]
      }
    ]
  },
  "stats_config": {
    "stats_tags": [
      {
        "regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.custom_hash"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.service_subset"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.service"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.namespace"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.datacenter"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.routing_type"
      },
      {
        "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)",
        "tag_name": "consul.trust_domain"
      },
      {
        "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)",
        "tag_name": "consul.target"
      },
      {
        "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)",
        "tag_name": "consul.full_target"
      },
      {
        "tag_name": "local_cluster",
        "fixed_value": "httpecho"
      }
    ],
    "use_all_default_tags": true
  },
  "dynamic_resources": {
    "lds_config": {
      "ads": {}
    },
    "cds_config": {
      "ads": {}
    },
    "ads_config": {
      "api_type": "GRPC",
      "grpc_services": {
        "initial_metadata": [
          {
            "key": "x-consul-token",
            "value": "dc612e72-e2cd-dc8c-b9d2-53315ae41258"
          }
        ],
        "envoy_grpc": {
          "cluster_name": "local_agent"
        }
      }
    }
  }
}
vagrant@app01:~$
banks commented 4 years ago

@allthingsclowd thanks for the great report. For future ref, it's better to leave issues open - they'll be closed when the PR gets merged and makes sure they stay visible on our project milestones etc.