fluent / fluent-bit-docker-image

Docker image for Fluent Bit
https://hub.docker.com/r/fluent/fluent-bit/
Apache License 2.0
67 stars 75 forks source link

How to do health check on docker env. (AWS ECS)? #34

Open JoHuang opened 4 years ago

JoHuang commented 4 years ago

According to AWS doc: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_HealthCheck.html [ "CMD-SHELL", "curl -f http://localhost/ || exit 1" ] is the simplest way to do health check

However there is no shell and no curl in the image, so I can get the container healthy. No package manager to install curl. I tried debug version (busybox) which contains wget to replace curl, but there is still no shell to execute it.

I would like to how to do health check on docker (AWS ECS)? Thanks

P.S. I would like to use fluent-bit as a sidecar container of my node.js app, so I have to make sure fluent-bit become healthy first, then start node.js app container. That's why I have to implement health check on AWS ECS.

brewkode commented 4 years ago

I used the debug image and configured it with the below HEALTHCHECK to test out locally, it works. About to test on ECS. Shall keep this place updated!

HEALTHCHECK --interval=5s --timeout=5s --retries=3 CMD ["busybox", "wget", "-q", "-O", "-", "http://localhost/"]
mshetland-demosphere commented 2 years ago

If anyone is interested, found solution for health

First you need to turn on healthcheck for the service -> see https://docs.fluentbit.io/manual/administration/monitoring#health-check-for-fluent-bit

Then you can setup health check with be HEALTHCHECK --interval=10s --start-period=10s --retries=5 --timeout=2s CMD ["/wget", "--quiet", "--output-document=-", "http://localhost:2020/api/v1/health"]

Now for the health check to work you need wget, which I simply pull from google distroless image, here is simple docker image

FROM gcr.io/distroless/base:debug as distroless-base

FROM fluent/fluent-bit:1.9.1

# Add WGET for healthcheck
COPY --from=distroless-base ["/busybox/wget", "/wget"]

HEALTHCHECK --interval=10s --start-period=10s --retries=5 --timeout=2s CMD ["/wget", "--quiet", "--output-document=-", "http://localhost:2020/api/v1/health"]
PettitWesley commented 1 year ago

@JoHuang @mshetland-demosphere @brewkode AWS has created an ECS FireLens/Fluent Bit health check tutorial that provides multiple options and discusses their tradeoffs: https://github.com/aws-samples/amazon-ecs-firelens-examples/tree/mainline/examples/fluent-bit/health-check