search

bitnami / containers

Bitnami container images
https://bitnami.com
Other
3.36k stars 4.83k forks source link

[bitnami/fluentd] could not install geoip #55364

Closed gulecroc closed 8 months ago

gulecroc commented 9 months ago

Name and Version

bitnami/fluentd:1.16.3-debian-11-r3

What architecture are you using?

None

What steps will reproduce the bug?

Trying to install fluentd geiop filter (https://docs.fluentd.org/filter/geoip) with bitnami/fluentd chart.

This plugin has prerequires : https://docs.fluentd.org/filter/geoip#prerequisites

I tried :

aggregator:
  extraGems:
    - fluent-plugin-geoip

Init-container install-extra-gems errors :

Building native extensions. This could take a while...
ERROR:  Error installing fluent-plugin-geoip:
    ERROR: Failed to build gem native extension.

    current directory: /opt/bitnami/fluentd/gems/geoip-c-0.9.1/ext/geoip
/opt/bitnami/ruby/bin/ruby extconf.rb
checking for iconv_open() in iconv.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/opt/bitnami/ruby/bin/$(RUBY_BASE_NAME)
    --with-geoip-dir
    --without-geoip-dir
    --with-geoip-include
    --without-geoip-include=${geoip-dir}/include
    --with-geoip-lib
    --without-geoip-lib=${geoip-dir}/lib
/opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:490:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:583:in `try_link0'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:601:in `try_link'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:812:in `try_func'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:1116:in `block in have_func'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:989:in `block in checking_for'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:354:in `block (2 levels) in postpone'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:324:in `open'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:354:in `block in postpone'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:324:in `open'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:350:in `postpone'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:988:in `checking_for'
    from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:1115:in `have_func'
    from extconf.rb:8:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /opt/bitnami/fluentd/extensions/x86_64-linux/3.1.0-static/geoip-c-0.9.1/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /opt/bitnami/fluentd/gems/geoip-c-0.9.1 for inspection.
Results logged to /opt/bitnami/fluentd/extensions/x86_64-linux/3.1.0-static/geoip-c-0.9.1/gem_make.out
FROM bitnami/fluentd:1.16.3-debian-11-r3

USER root
RUN install_packages build-essential libgeoip-dev

Use my image in values aggregator.image, but same errors in init-container install-extra-gems

FROM bitnami/fluentd:1.16.3-debian-11-r3

USER root
RUN install_packages build-essential libgeoip-dev
RUN fluent-gem install fluent-plugin-geoip --no-document

Build is KO :

$ docker build .
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  45.06kB
Step 1/4 : FROM bitnami/fluentd:1.16.3-debian-11-r3
 ---> 0aa99db2a0bd
Step 2/4 : USER root
 ---> Using cache
 ---> 14f5dfa4db94
Step 3/4 : RUN install_packages build-essential libgeoip-dev
 ---> Using cache
 ---> cb67b462bb9a
Step 4/4 : RUN fluent-gem install fluent-plugin-geoip --no-document
 ---> Running in 8c9c7431933e
Building native extensions. This could take a while...
ERROR:  Error installing fluent-plugin-geoip:
        ERROR: Failed to build gem native extension.

    current directory: /opt/bitnami/fluentd/gems/geoip-c-0.9.1/ext/geoip
/opt/bitnami/ruby/bin/ruby extconf.rb
checking for iconv_open() in iconv.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/opt/bitnami/ruby/bin/$(RUBY_BASE_NAME)
        --with-geoip-dir
        --without-geoip-dir
        --with-geoip-include
        --without-geoip-include=${geoip-dir}/include
        --with-geoip-lib
        --without-geoip-lib=${geoip-dir}/lib
/opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:490:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:583:in `try_link0'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:601:in `try_link'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:812:in `try_func'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:1116:in `block in have_func'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:989:in `block in checking_for'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:354:in `block (2 levels) in postpone'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:324:in `open'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:354:in `block in postpone'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:324:in `open'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:350:in `postpone'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:988:in `checking_for'
        from /opt/bitnami/ruby/lib/ruby/3.1.0/mkmf.rb:1115:in `have_func'
        from extconf.rb:8:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /opt/bitnami/fluentd/extensions/x86_64-linux/3.1.0-static/geoip-c-0.9.1/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /opt/bitnami/fluentd/gems/geoip-c-0.9.1 for inspection.
Results logged to /opt/bitnami/fluentd/extensions/x86_64-linux/3.1.0-static/geoip-c-0.9.1/gem_make.out
The command '/bin/bash -o errexit -o nounset -o pipefail -c fluent-gem install fluent-plugin-geoip --no-document' returned a non-zero code: 1

What is the expected behavior?

Install fluentd geoip gem with success since prerequires are packaged in fluend image

What do you see instead?

Command fluent-gem install fluent-plugin-geoip is in error

Additional information

No response

gulecroc commented 9 months ago

Thank to this issue : https://github.com/y-ken/fluent-plugin-geoip/issues/100

I extend the bitnami/fluentd image :

FROM bitnami/fluentd:1.16.3-debian-11-r3

USER root
RUN install_packages build-essential libgeoip-dev libmaxminddb-dev automake autoconf libtool zlib1g-dev

USER 1001

The geoip plugin is successfully built from bitnami/fluentd chart :

aggregator:
  extraGems:
    - fluent-plugin-geoip

The command behind this configuration is (executed as root) : https://github.com/bitnami/charts/blob/main/bitnami/fluentd/templates/aggregator-statefulset.yaml

cd /tmp

# install extra gems
fluent-gem install fluent-plugin-geoip --no-document

# copy installed gems to shared dir
cp -r /opt/bitnami/fluentd/gems/* /gems/
cp -r /opt/bitnami/fluentd/specifications/* /specifications/

When starting fluentd, the plugin is listed but the extensions does not work:

2024-01-26T14:34:20.184717082Z Ignoring geoip-c-0.9.1 because its extensions are not built. Try: gem pristine geoip-c --version 0.9.1
2024-01-26T14:34:20.185449192Z Ignoring geoip2_c-0.3.4 because its extensions are not built. Try: gem pristine geoip2_c --version 0.3.4
...
2024-01-26 14:34:20 +0000 [info]: gem 'fluent-plugin-geoip' version '1.3.2'
gulecroc commented 8 months ago

I find extensions directory in /opt/bitnami/fluentd/extensions.

The directory content is needed for geoip plugin.

I close this issue and open a PR for bitnami/fluentd chart.