search

rabbitmq / rabbitmq-management

RabbitMQ Management UI and HTTP API
https://www.rabbitmq.com/management.html
Other
370 stars 165 forks source link

Failed healthchecks return HTTP 200 status #840

Closed coderanger closed 3 years ago

coderanger commented 4 years ago

The healthcheck APIs return a 200 response status even when the check has failed. This makes them harder to use as a 2xx response is generally considered a pass and a non-2xx a fail by most systems. These APIs should return a 500 response code on failure.

michaelklishin commented 4 years ago

FIY, this One True Health Check™ endpoint is deprecated.

coderanger commented 4 years ago

Interesting, I had kind of assumed it was doing the same checks under the hood. Executable checks do exist in Kubernetes but they are much higher overhead and cause issues if the main container process doesn't have a handler for SIGCHILD.

coderanger commented 4 years ago

To answer my unstated question though: tracing through what is actually running in the default rabbitmq container image, PID 1 is a copy of erl which doesn't react to SIGCHILD but does call waitpid as part of its loop, though I don't think it's for this purpose. So mildly by accident I think this should work okay.

michaelklishin commented 4 years ago

The new fine-grained health checks will be added one day (contributions are welcome). This One True Health Check™ endpoint (and CLI command) is very intrusive (asks every channel and queue replica in the system to emit some stats on the spot), almost guaranteed to produce false positives under heavy load, and more importantly, users do not understand what it really does.

There is no One True Health Check™ as different teams define what constitutes "healthy" for their specific environment entirely differently. Instead, a set of smaller, composable checks was introduced so you can choose what makes sense to you.

We will consider #841 but the entire approach is problematic so instead the efforts should go into making individual CLI health checks available as individual HTTP API endpoints.