hashicorp / consul-k8s

First-class support for Consul Service Mesh on Kubernetes
https://www.consul.io/docs/k8s
Mozilla Public License 2.0
669 stars 322 forks source link

Deregister services after unhealthy #1825

Open javiermendozain opened 1 year ago

javiermendozain commented 1 year ago

Community Note


Overview of the Issue

Somebody had the same situation?

Reproduction Steps

Scenario: consul image: 1.14.2, using 3 server We have terragrunt and terraform, using the default value.yaml, and replace those properties👇:

{
"server.replicas" = 3
"client.enabled" = false
"connectInject.enabled" = false
"syncCatalog.enabled" = false
"global.acls.manageSystemACLs" = false
"global.acls.nodeSelecor" = "group:kafka"
}

When: we scale down the microservices massively (ej: 10 to 1 pod ), some instance services don't deregister

Logs

image

image

image

The consul server not found the service image

2023-01-10T20:52:07.135Z [ERROR] agent.http: Request error: method=PUT url="/v1/agent/service/deregister/ef52afee-c25e-464a-a0e1-32df4c0d6878?merge-central-config=&near=_agent&node-meta=sd-consul-server-1%3Asd-consul-server-1" from=172.24.13.235:49322 error="Unknown service ID \"ef52afee-c25e-464a-a0e1-32df4c0d6878\". Ensure that the service ID is passed, not the service name." 2023-01-10T20:52:07.158Z [ERROR] agent.http: Request error: method=PUT url="/v1/agent/service/deregister/ef52afee-c25e-464a-a0e1-32df4c0d6878?merge-central-config=&near=_agent&node-meta=sd-consul-server-1%3Asd-consul-server-1" from=172.24.13.235:49322 error="Unknown service ID \"ef52afee-c25e-464a-a0e1-32df4c0d6878\". Ensure that the service ID is passed, not the service name."

Expected behavior

Environment details

Consul: 1.14.2 the microservices are using consul client on Golang (v1.15.2) and .NET (1.6.10.8)

Additional Context

Juandavi1 commented 1 year ago

Hi ! Same here !

javiermendozain commented 1 year ago

Hi @blake @david-yu, @ishustava Hope you are so well , I would to know if it is clear the issue that I've commented on or do you have some recommendations for this one.?

MageshSrinivasulu commented 2 months ago

Facing similar issue