puppetlabs / puppetserver

Server automation framework and application
https://tickets.puppetlabs.com/browse/SERVER
Apache License 2.0
292 stars 235 forks source link

/status/v1/simple timing out regularly #2843

Open Alives opened 5 months ago

Alives commented 5 months ago

Describe the Bug

Queries to /status/v1/simple usually return in less than 1 second, but at times they do not return at all and timeout.

I assume a timeout is a catastrophic service check, but it would be nice to get some insight into what is happening here. It seems timeouts happen even when the service is responding to requests in a healthy manner.

Expected Behavior

Rapid and accurate responses according to the state of the service and stack for load balancers and state tracking to monitor.

Steps to Reproduce

  1. Start puppetserver 7.x stack
  2. Send traffic to it (in my case, 3 clients connecting per second)
  3. Keep checking /status/v1/simple
  4. Notice that at times it just won't respond, other times the responses can take 10s of seconds or more
  5. Also notice that regardless of the reported health, the service is fine, even if the healthcheck times out.

Environment