search

atomita / fluent-plugin-aws-elasticsearch-service

This repository is extends of https://github.com/uken/fluent-plugin-elasticsearch/ which made connectable to Amazon Elassticsearch Service using Aws Signers V4. Please check the requirements to connect to Amazon Elassticsearch Service without using Aws Signers V4 for before starting use. (reference https://groups.google.com/forum/#!msg/fluentd/uW87VAOqxeE/cfhenicEBAAJ)
MIT License
122 stars 49 forks source link

Gem install is slow #48

Closed castro1688 closed 5 years ago

castro1688 commented 6 years ago

As it stands fluent-plugin-aws-elasticsearch-service requires the aws-sdk gem. This includes 30+ aws-sdk-* items.

With sdk version 3 there is the ability to install specific modules. https://github.com/aws/aws-sdk-ruby/blob/master/README.md

dpgaspar commented 5 years ago

Same problem here!

stefaneg commented 5 years ago

Slow as in nothing happening for about 5 minutes. I would actually call that very close to not working at all.

Dockerfile:

FROM fluent/fluentd:v1.2.2

RUN apk add --update --virtual .build-deps \
        sudo build-base ruby-dev \
 && sudo gem install \
        fluent-plugin-aws-elasticsearch-service  \
        fluent-plugin-parser \
        fluent-plugin-kubernetes_metadata_filter \
 && sudo gem sources --clear-all \
 && apk del .build-deps \
 && rm -rf /var/cache/apk/* \
        /home/fluent/.gem/ruby/2.3.0/cache/*.gem

COPY ./conf/fluent.conf /fluentd/etc/fluent.conf

USER root

COPY ./entrypoint.sh /bin/entrypoint.sh
RUN chmod +x /bin/entrypoint.sh
thadamski commented 5 years ago

I too had/have the slow gem install issue. I was installing via chef with the td-agent cookbook, and I was able to get them gem installing fast by reducing the aws dependencies (as @castro1688 pointed out) to only what was needed (aws-sdk-core), rebuilding the gem, and then doing a local install. If you would like to do the same, check out the PR description (you'll probably want to bump the version in the .gemspec first): https://github.com/atomita/fluent-plugin-aws-elasticsearch-service/pull/49