search

fluent / fluent-plugin-kafka

Kafka input and output plugin for Fluentd
Other
303 stars 176 forks source link

Support AWS MSK Authentication #472

Open nonamef opened 1 year ago

nonamef commented 1 year ago

Is your feature request related to a problem? Please describe.

Currently using fluent with AWS MSK as unauthenticated with security groups to control access. Want to enable AWS IAM assume role for cross account access.

Describe the solution you'd like

Same setup as fluent-plugin-s3 for assume_role_credentials would be good. e.g https://github.com/fluent/fluent-plugin-s3/blob/master/docs/credentials.md

Describe alternatives you've considered

Only alternative is to continue as unauthenticated.

Additional context

No response

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

raytung commented 1 year ago

Hey @nonamef We have added AWS IAM auth support with access keys and secret keys. Unfortunately, since the ruby-kafka gem is deprecated, they are not adding support for assume roles. Looks like librdkafka is also not supporting AWS IAM auth so our hands are a bit tied on this one.