search

kiwigrid / helm-charts

Helm charts for Kubernetes curated by Kiwigrid
https://kiwigrid.github.io
MIT License
184 stars 210 forks source link

fluentd-elasticsearch additionalPlugins #326

Closed alankwon closed 4 years ago

alankwon commented 4 years ago

Is this a request for help?: Yes

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Version of Helm and Kubernetes: Helm 2.13.1 Kubernetes 1.15.5

Which chart in which version: helm-charts/charts/fluentd-elasticsearch/

What happened: I am trying to install an additional plugin, i.e. fluent-plugin-mongo. I issued the following command:

$ helm install --name fluentd --namespace logging kiwigrid/fluentd-elasticsearch --set elasticsearch.host=elasticsearch-client.logging.svc.cluster.local,additionalPlugins[0].name=fluent-plugin-mongo,additionalPlugins[0].version=1.3.0

What you expected to happen: Expected fluent-plugin-mongo-1.3.0 and mongo-2.6.4 gems to be installed in the pod, but getting this error:

$ kubectl logs -n logging fluentd-fluentd-elasticsearch-5b5b7 Building native extensions. This could take a while... ERROR: Error installing fluent-plugin-mongo: ERROR: Failed to build gem native extension.

current directory: /usr/local/bundle/gems/bson-4.8.2/ext/bson

/usr/local/bin/ruby -I /usr/local/lib/ruby/2.7.0 -r ./siteconf20200415-7-19o5a0l.rb extconf.rb creating Makefile

current directory: /usr/local/bundle/gems/bson-4.8.2/ext/bson make "DESTDIR=" clean sh: 1: make: not found

current directory: /usr/local/bundle/gems/bson-4.8.2/ext/bson make "DESTDIR=" sh: 1: make: not found

make failed, exit code 127

Gem files will remain installed in /usr/local/bundle/gems/bson-4.8.2 for inspection. Results logged to /usr/local/bundle/extensions/x86_64-linux/2.7.0/bson-4.8.2/gem_make.out

How to reproduce it (as minimally and precisely as possible): Just run the command specified above.

Anything else we need to know: I found similar problem being described in the fluend documentation: https://docs.fluentd.org/deployment/plugin-management#gem-and-native-extension

I tried to launch the pod without the additionalPlugin, got a shell to the container with kubectl exec and ran the following:

$ apt-get install ruby-dev gcc make

Then fluent-gem install fluent-plugin-mongo went through.

monotek commented 4 years ago

the dockerfile uses builder pattern now. see: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image/Dockerfile

so build tools are not available anymore in the image.

workaround would be to use your own image, which has all the stuff you want included.

question is, if its till makes sense to have the plugin option in the helm chart. i guess not... its bad practice anyway to change your docker image on runtime...

alankwon commented 4 years ago

Thanks for the response.

I noticed several older issues asking for other plugins to be included and saw that the additionalPlugins was added. I don't necessarily think it's bad to have this option, if it works reliably. It's not quite the same, but Helm does allow some manipulation with things like hooks. Is there a particular reason why the build tools are removed?

monotek commented 4 years ago

They are not needed if you use the container as is and therefore removing them decreases the attack surface.

chadlwilson commented 4 years ago

question is, if its till makes sense to have the plugin option in the helm chart. i guess not... its bad practice anyway to change your docker image on runtime...

Agree that this is bad practice, as is including dev tools in the images - my vote would be to remove the additionalPlugins support; especially as most of the available fluentd images appear to have too high an attack surface as it is.

monotek commented 4 years ago

Option will be removed in: https://github.com/kiwigrid/helm-charts/pull/329