search

envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.72k stars 4.75k forks source link

Memory leak on health checking #5133

Closed kindermoumoute closed 5 years ago

kindermoumoute commented 5 years ago

Memory leak on health checking

Description: I'm having a memory leak on production: image

After running the heap profiler I could clearly see the leak comes from health checking:

image

The same heap subtracted from the original heap:

image

I could not reproduce the issue locally yet, that's why I'm not posting my config here.

Heap profiles: envoy-heaps.zip

repokitteh-read-only[bot] commented 5 years ago

:scream_cat: Error while processing event:

evaluation error
error: finished: cannot load github.com/repokitteh/modules/lib/circleci.star: error from server: {module load error GET https://api.github.com/repos/repokitteh/modules/contents/lib/circleci.star: 401 Bad credentials [] map[]}
:cat: Caused by: https://github.com/envoyproxy/envoy/issues/5133 was labeled by @mattklein123. see: [more](https://github.com/envoyproxy/envoy/issues/5133), [trace](https://prod.repokitteh.app/traces/ui/envoyproxy/envoy/d380c880-f253-11e8-927e-b7ac72c60f13).
mattklein123 commented 5 years ago

What version are you running? What is your approximate config? There was an issue with unbounded HC hosts for DNS clusters that has been fixed.

kindermoumoute commented 5 years ago

What version are you running?

Docker container envoyproxy/envoy-alpine-debug:v1.8.0

What is your approximate config?

There is my approximate config (anonymised config dump). ```json { "configs": [ { "@type": "type.googleapis.com/envoy.admin.v2alpha.BootstrapConfigDump", "bootstrap": { "node": { "id": "vtydc2-p12pub-lb01", "cluster": "myproject-mesh", "build_version": "5d25f466c3410c0dfa735d7d4358beb76b2da507/1.8.0/Clean/RELEASE" }, "static_resources": { "listeners": [ { "name": "Mayme Fitzgerald", "address": "1134 Akda Point", "filter_chains": [ { "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Lee Fox" } ] } }, "filters": [ { "name": "envoy.http_connection_manager", "config": { "server_name": "myproject_mesh_grpc", "access_log": { "name": "Katharine Glover", "config": { "path": "/dev/stdout" } }, "http_filters": [ { "name": "envoy.health_check", "config": { "pass_through_mode": false, "headers": [ { "name": ":path", "exact_match": "/health" } ] } }, { "name": "Hester Cohen", "config": { "grpc_service": { "envoy_grpc": { "cluster_name": "myproject_auth" }, "timeout": "30s" } } }, { "name": "Billy Bailey", "config": {} } ], "route_config": { "name": "Olivia Schwartz", "virtual_hosts": [ { "domains": [ "*" ], "name": "Edward Morales", "routes": [ { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Mrs.", "grpc": {} } }, { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Mrs.", "grpc": {} } }, { "match": { "prefix": "Mr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Mr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Miss", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Dr.", "grpc": {} } }, { "route": { "cluster": "serviceXXX" }, "match": { "grpc": {}, "prefix": "Dr." } }, { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Dr.", "grpc": {} } }, { "route": { "cluster": "serviceXXX" }, "match": { "grpc": {}, "prefix": "Dr." } } ] } ] }, "codec_type": "http2", "stat_prefix": "myproject_mesh_grpc", "use_remote_address": false } } ] } ] }, { "name": "Alma Murray", "address": "302 Cake Boulevard", "filter_chains": [ { "tls_context": { "common_tls_context": { "tls_certificate_sds_secret_configs": [ { "name": "Jean Palmer" } ], "validation_context_sds_secret_config": { "name": "Michael Stanley" } }, "require_client_certificate": true }, "filters": [ { "name": "envoy.redis_proxy", "config": { "settings": { "op_timeout": "10s" }, "stat_prefix": "myproject_redis", "cluster": "myproject_redis" } } ] } ] } ], "clusters": [ { "name": "Mark Parsons", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "myproject-auth" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Roger Waters" } ], "validation_context_sds_secret_config": { "name": "Milton Barber" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "myproject_auth", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "748 Remop Junction" } }, { "endpoint": { "address": "1356 Gucuju Highway" } } ] } ] } }, { "name": "Richard Robbins", "type": "STRICT_DNS", "connect_timeout": "30s", "load_assignment": { "cluster_name": "myproject_mesh_exporter", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1006 Goka Pass" } } ] } ] } }, { "name": "Corey Adkins", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "custom_health_check": { "name": "Raymond Gordon" } } ], "tls_context": { "common_tls_context": { "tls_certificate_sds_secret_configs": [ { "name": "Mittie Powers" } ], "validation_context_sds_secret_config": { "name": "Alan Bishop" } } }, "load_assignment": { "cluster_name": "myproject_redis", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1091 Mapu Way" } } ], "priority": 1 }, { "lb_endpoints": [ { "endpoint": { "address": "1112 Tamet Road" } } ] } ] } }, { "name": "Lucas Fernandez", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Millie Park" } ], "validation_context_sds_secret_config": { "name": "Lily Frank" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "155 Oziewi Grove" } }, { "endpoint": { "address": "1389 Puug Place" } } ] } ] } }, { "name": "Nellie Jenkins", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Raymond Price" } ], "validation_context_sds_secret_config": { "name": "Helen Blake" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1162 Nici Avenue" } }, { "endpoint": { "address": "43 Jikmo Street" } }, { "endpoint": { "address": "380 Dahoz Square" } }, { "endpoint": { "address": "1994 Fulod Circle" } } ] } ] } }, { "name": "Callie Newton", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Cordelia Ward" } ], "validation_context_sds_secret_config": { "name": "Walter Copeland" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "45 Butbap Glen" } }, { "endpoint": { "address": "1310 Milcur River" } } ] } ] } }, { "name": "Allen Chambers", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Ruth Holloway" } ], "validation_context_sds_secret_config": { "name": "Florence Valdez" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1620 Joeho Terrace" } }, { "endpoint": { "address": "1636 Polupu Circle" } }, { "endpoint": { "address": "326 Tohuv Circle" } }, { "endpoint": { "address": "1500 Dotfa Extension" } }, { "endpoint": { "address": "1163 Zokzo Heights" } }, { "endpoint": { "address": "805 Sushid Glen" } }, { "endpoint": { "address": "430 Kasken Point" } }, { "endpoint": { "address": "549 Jiced Square" } }, { "endpoint": { "address": "244 Tipwel Pass" } }, { "endpoint": { "address": "398 Zecas Point" } }, { "endpoint": { "address": "1519 Zigrer Heights" } } ] } ] } }, { "name": "Susie McCormick", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Ernest Casey" } ], "validation_context_sds_secret_config": { "name": "Dora Richards" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "152 Fiwe Grove" } } ] } ] } }, { "name": "Lelia Myers", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Hester Stevens" } ], "validation_context_sds_secret_config": { "name": "Kenneth Harper" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1006 Olam Extension" } } ] } ] } }, { "name": "Francis Erickson", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Estella Martin" } ], "validation_context_sds_secret_config": { "name": "Mabel Webster" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "97 Neka Ridge" } }, { "endpoint": { "address": "622 Muduga Way" } } ] } ] } }, { "name": "Clifford Alvarado", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Thomas White" } ], "validation_context_sds_secret_config": { "name": "Leonard Austin" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "117 Kanah Ridge" } }, { "endpoint": { "address": "545 Puuhe Road" } } ] } ] } }, { "name": "Annie Gill", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Verna Curry" } ], "validation_context_sds_secret_config": { "name": "Duane Nichols" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "213 Soroz Extension" } }, { "endpoint": { "address": "1738 Zogod Manor" } } ] } ] } } ], "secrets": [ { "name": "Albert Cain", "tls_certificate": { "certificate_chain": { "filename": "/data/ssl/myproject.crt" }, "private_key": { "filename": "/data/ssl/myproject.key" } } }, { "name": "Rose Jennings", "tls_certificate": { "certificate_chain": { "filename": "/data/ssl/wildcard.pv.ocshq.com.crt" }, "private_key": { "filename": "/data/ssl/wildcard.pv.ocshq.com.key" } } }, { "name": "Ian Conner", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "myproject-auth" ] } }, { "name": "Annie McBride", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "myproject-auth" ] } }, { "name": "Delia Fox", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "myproject-redis" ] } }, { "name": "Lilly Wheeler", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Ray Stone", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Jean Dunn", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Edna Miller", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Sally Beck", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Mattie Mathis", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Ralph Santiago", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Ola Hale", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } }, { "name": "Lucinda Hanson", "validation_context": { "trusted_ca": { "filename": "/data/ssl/myproject-ca.crt" }, "verify_subject_alt_name": [ "my.service" ] } } ] }, "stats_sinks": [ { "name": "Bryan Byrd", "config": { "tcp_cluster_name": "myproject_mesh_exporter" } } ], "admin": { "access_log_path": "/dev/null", "address": "1098 Ridak Highway" } }, "last_updated": "2018-11-22T12:34:18.782Z" }, { "@type": "type.googleapis.com/envoy.admin.v2alpha.ClustersConfigDump", "static_clusters": [ { "cluster": { "name": "Richard Vaughn", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "myproject-auth" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Esther Phelps" } ], "validation_context_sds_secret_config": { "name": "Grace Stone" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "myproject_auth", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1421 Gozik Manor" } }, { "endpoint": { "address": "1995 Hiob Glen" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.796Z" }, { "cluster": { "name": "Benjamin Maldonado", "type": "STRICT_DNS", "connect_timeout": "30s", "load_assignment": { "cluster_name": "myproject_mesh_exporter", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1951 Zele Point" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.797Z" }, { "cluster": { "name": "Maurice Neal", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "custom_health_check": { "name": "Sue Wilson" } } ], "tls_context": { "common_tls_context": { "tls_certificate_sds_secret_configs": [ { "name": "Henry Greene" } ], "validation_context_sds_secret_config": { "name": "Clara Guzman" } } }, "load_assignment": { "cluster_name": "myproject_redis", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "525 Pakbez Way" } } ], "priority": 1 }, { "lb_endpoints": [ { "endpoint": { "address": "330 Ocimo Point" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.801Z" }, { "cluster": { "name": "Harriet Joseph", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Tillie Tate" } ], "validation_context_sds_secret_config": { "name": "Mable Kim" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1695 Wulto Trail" } }, { "endpoint": { "address": "1546 Anew Junction" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.805Z" }, { "cluster": { "name": "Mollie Hardy", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Edwin Peterson" } ], "validation_context_sds_secret_config": { "name": "Helen Tran" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1118 Fapro Trail" } }, { "endpoint": { "address": "239 Rijuhu Ridge" } }, { "endpoint": { "address": "291 Dituc Ridge" } }, { "endpoint": { "address": "1120 Vozib Circle" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.809Z" }, { "cluster": { "name": "Sara Watts", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Alejandro Dunn" } ], "validation_context_sds_secret_config": { "name": "Mayme Tran" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "470 Wuro Drive" } }, { "endpoint": { "address": "652 Vasma Grove" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.813Z" }, { "cluster": { "name": "Roger Howard", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Cecelia Willis" } ], "validation_context_sds_secret_config": { "name": "Ophelia Beck" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "628 Zuver Path" } }, { "endpoint": { "address": "808 Lonmop Lane" } }, { "endpoint": { "address": "554 Lois Center" } }, { "endpoint": { "address": "1680 Vazusa Park" } }, { "endpoint": { "address": "1926 Tojan Highway" } }, { "endpoint": { "address": "386 Asoem Street" } }, { "endpoint": { "address": "1275 Wafoh Street" } }, { "endpoint": { "address": "1786 Vekfov Loop" } }, { "endpoint": { "address": "1661 Sugzub Place" } }, { "endpoint": { "address": "224 Celbes Terrace" } }, { "endpoint": { "address": "811 Tujed Heights" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.817Z" }, { "cluster": { "name": "Miguel Mack", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Matilda Clayton" } ], "validation_context_sds_secret_config": { "name": "Lura Larson" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "279 Mogbit Trail" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.821Z" }, { "cluster": { "name": "Sarah Moran", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Mathilda Gibson" } ], "validation_context_sds_secret_config": { "name": "Alberta Warren" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1295 Jibruj Parkway" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.825Z" }, { "cluster": { "name": "Peter Lewis", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Georgie Padilla" } ], "validation_context_sds_secret_config": { "name": "Jean Kennedy" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1173 Vaswun Street" } }, { "endpoint": { "address": "1211 Heflez Glen" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.829Z" }, { "cluster": { "name": "Jeremy Edwards", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Josephine Blake" } ], "validation_context_sds_secret_config": { "name": "Floyd Arnold" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "943 Kecpuc Circle" } }, { "endpoint": { "address": "492 Pigun Terrace" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.832Z" }, { "cluster": { "name": "Donald Griffith", "type": "STRICT_DNS", "connect_timeout": "30s", "health_checks": [ { "timeout": "10s", "interval": "15s", "unhealthy_threshold": 1, "healthy_threshold": 1, "grpc_health_check": { "service_name": "my.service" } } ], "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Lee Tate" } ], "validation_context_sds_secret_config": { "name": "Carl Hart" } } }, "http2_protocol_options": {}, "load_assignment": { "cluster_name": "serviceXXX", "endpoints": [ { "lb_endpoints": [ { "endpoint": { "address": "1693 Gajo Way" } }, { "endpoint": { "address": "1642 Suhpe Square" } } ] } ] } }, "last_updated": "2018-11-22T12:34:18.836Z" } ] }, { "@type": "type.googleapis.com/envoy.admin.v2alpha.ListenersConfigDump", "static_listeners": [ { "listener": { "name": "Lawrence Munoz", "address": "719 Aboun Boulevard", "filter_chains": [ { "tls_context": { "common_tls_context": { "alpn_protocols": [ "h2" ], "tls_certificate_sds_secret_configs": [ { "name": "Maude Perkins" } ] } }, "filters": [ { "name": "Bertie Rhodes", "config": { "server_name": "myproject_mesh_grpc", "access_log": { "name": "Mable Jackson", "config": { "path": "/dev/stdout" } }, "http_filters": [ { "name": "Brian Herrera", "config": { "headers": [ { "exact_match": "/health", "name": "Danny Wilkins" } ], "pass_through_mode": false } }, { "name": "Beulah Logan", "config": { "grpc_service": { "envoy_grpc": { "cluster_name": "myproject_auth" }, "timeout": "30s" } } }, { "name": "Lou Hayes", "config": {} } ], "route_config": { "name": "Phillip Malone", "virtual_hosts": [ { "routes": [ { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Mrs.", "grpc": {} } }, { "match": { "prefix": "Mr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "route": { "cluster": "serviceXXX" }, "match": { "grpc": {}, "prefix": "Mrs." } }, { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Dr.", "grpc": {} } }, { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Mrs.", "grpc": {} } }, { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Dr.", "grpc": {} } }, { "route": { "cluster": "serviceXXX" }, "match": { "prefix": "Mr.", "grpc": {} } }, { "match": { "prefix": "Mrs.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "route": { "cluster": "serviceXXX" }, "match": { "grpc": {}, "prefix": "Mrs." } } ], "domains": [ "*" ], "name": "Richard Lowe" } ] }, "codec_type": "http2", "stat_prefix": "myproject_mesh_grpc", "use_remote_address": false } } ] } ] }, "last_updated": "2018-11-22T12:34:18.865Z" }, { "listener": { "name": "Cecilia Haynes", "address": "975 Osda Lane", "filter_chains": [ { "tls_context": { "common_tls_context": { "tls_certificate_sds_secret_configs": [ { "name": "Ralph King" } ], "validation_context_sds_secret_config": { "name": "Luke Baldwin" } }, "require_client_certificate": true }, "filters": [ { "name": "Sue Reynolds", "config": { "stat_prefix": "myproject_redis", "cluster": "myproject_redis", "settings": { "op_timeout": "10s" } } } ] } ] }, "last_updated": "2018-11-22T12:34:18.872Z" } ] }, { "@type": "type.googleapis.com/envoy.admin.v2alpha.RoutesConfigDump", "static_route_configs": [ { "route_config": { "name": "Eula Waters", "virtual_hosts": [ { "name": "Herbert Davis", "domains": [ "*" ], "routes": [ { "match": { "prefix": "Mr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Mr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Dr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Mrs.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Dr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Miss", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Dr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Mr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } }, { "match": { "prefix": "Dr.", "grpc": {} }, "route": { "cluster": "serviceXXX" } } ] } ] }, "last_updated": "2018-11-22T12:34:18.855Z" } ] } ] } ```
mattklein123 commented 5 years ago

This is the issue I was talking about, but that fix made it into 1.8.0: https://github.com/envoyproxy/envoy/issues/4548. Looks like this might be an issue with the gRPC health checker. Will need some investigation.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

stale[bot] commented 5 years ago

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.