search

fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.89k stars 1.59k forks source link

Publish to ECR public galleries #9176

Open acm19 opened 3 months ago

acm19 commented 3 months ago

Is your feature request related to a problem? Please describe. We run our workloads in ECS/EKS, as many people do. We use Fluent Bit as sidecar container to push logs to different backends depending on the app. We need a reliable source to pull the image from every time a new container spins off. So basically we mostly only use ECR deployed images, as many people that need reliable image pull running in AWS.

Describe the solution you'd like I'd like to be able to use the official Fluent Bit image, but fetch it from a public ECR Gallery.

Describe alternatives you've considered At the moment we're using this image: https://gallery.ecr.aws/bitnami/fluent-bit. But we'd rather use the official image, which is more compact and secure since that one uses a debian base.

Additional context There was a similar request in the past. But it's different in the sense we want it to be used for production deployments where a third party isn't relied on to start up a new container.

patrick-stephens commented 3 months ago

Technically this is fairly easy to do at release time, we already sync the staging image to multiple locations: https://github.com/fluent/fluent-bit/blob/f54b370cd90c543e8a964e9b579d2aec437ee23b/.github/workflows/staging-release.yaml#L404-L437

Main thing to handle is authentication and whether there is a better approach, e.g. get mirrors set up by AWS to just pull latest image from ghcr.io.

There is also the AWS supplied image too but this is based on 1.9.

acm19 commented 3 months ago

There is also the AWS supplied image too but this is based on 1.9.

I want to explicitly avoid that one, it comes with custom AWS stuff, consumes more memory and as you say it doesn't follow Fluent Bit versions / delivery cycle.

e.g. get mirrors set up by AWS to just pull latest image from ghcr.io.

I'll look into Pull Through Cache to work around it, but even if it works, I think having it public would help production workload to work out of the box in AWS.

patrick-stephens commented 2 months ago

Yup, I agree but main thing is where to put it? I can update CI quite easily once we have that and creds for it.