Open thisfred opened 2 months ago
We're working on EKS Pod identity support, which involves refactoring the HTTP Provider. We're adding support for this env var in the same feature: https://github.com/fluent/fluent-bit/pull/8826
Please take a look.
AWS Distro PR diff: https://github.com/PettitWesley/fluent-bit/pull/32
These images contain the feature support along with some other upcoming work we are testing:
# Base image
144718711470.dkr.ecr.us-west-2.amazonaws.com/aws-for-fluent-bit:grace-input-chunk-check
# Image with Fluent Bit ECS Init helper added:
# https://github.com/aws-samples/amazon-ecs-firelens-examples/tree/mainline?tab=readme-ov-file#aws-for-fluent-bit-init-tag-examples
144718711470.dkr.ecr.us-west-2.amazonaws.com/aws-for-fluent-bit:init-grace-input-chunk-check
We are trying to use FluentBit in an AWS IoT Greengrass Component to log to Cloudwatch. The way Greengrass Components authenticate to AWS services is through a Token Exchange Service (which is another Greengrass component) running on the same device. In order to have code using the AWS SDKs be able to ask this service for credentials, an environment variable named
AWS_CONTAINER_CREDENTIALS_FULL_URI
can be set. Unfortunately aws-for-fluent-bit (or at least the cloudwatch_logs plugin part of it) does not seem to detect the presence of this variable, and instead only looks forAWS_CONTAINER_CREDENTIALS_RELATIVE_URI
, which doesn't work for the purposes of using anything running on the local device instead of in AWS.Example invocation with environment variables passed:
fluent-bit/bin/fluent-bit -i cpu -o cloudwatch_logs -p region=us-east-1 -p log_group_name=group_name -p log_stream_prefix=stream_prefix -p role_arn=[REDACTED] -vvv
Relevant log output:
Running commands in the same container that use the AWS SDK (like a boto request to S3) do successfully detect and use the
AWS_CONTAINER_CREDENTIALS_FULL_URI
environment variable, and are able to aqcuire the necessary credentials to talk to AWS services.