fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.73k stars 1.56k forks source link

Splunk HEC token should take a file path #7013

Open teancom opened 1 year ago

teancom commented 1 year ago

Is your feature request related to a problem? Please describe. Our secrets are kept in an external secret keeper, and we use an init container to grab them and write them out on disk on container startup. There is no automated way to scrape those files and turn them into environment variables. When using fluentd, we can do the following:

 hec_token "#{File.open('/secrets/app/SPLUNK_HEC_TOKEN').read.strip}"

But fluent-bit does not support either inline Ruby (of course) or the ability to point to a file and have the token read from it.

Describe the solution you'd like Two possible solutions spring to mind: ) auto-detect that the token starts with an / and treat it as a path (not a big fan of this) ) Add another config key like Splunk_Token_Path that can be used in place of Splunk_Token

Describe alternatives you've considered @agup006 had the suggestion of using the -debug container, which contains a shell and would allow us to use a small script to grab the file, export it, and then start fluent-bit. However the debug containers are much larger than the non-debug, containing an entire linux distribution. And I would prefer not to change the Entrypoint of the container.

Additional context We are trying to follow https://blog.forcesunseen.com/stop-storing-secrets-in-environment-variables in our approach to k8s secret keeping, which is why we don't have our secret-keeper automatically convert them to environment variables. Just for context as to why this is an issue in the first place.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

teancom commented 1 year ago

Should things tagged feature-request be automatically exempt-stale as well? Or is this a way to automated-ly check in to see if the feature is still desired? (It is, btw).

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

teancom commented 1 year ago

I continue to think this would be nifty.

github-actions[bot] commented 9 months ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

teancom commented 9 months ago

This would still be great

github-actions[bot] commented 5 months ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

teancom commented 5 months ago

This would still be super neat.

ahothan commented 4 months ago

+1 You've been tenacious @teancom ;-) any way to upvote this feature request?

github-actions[bot] commented 1 month ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

teancom commented 1 month ago

Up VOTE up VOTE up VOTE!