ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "faraday"

[blakejarvis]

Faraday version error when attempting to install the logstash-filter-greynoise plugin.

I am running logstash in a docker container as a part of the docker-elk deployment, and I have attempted to install the greynoise plugin both at the creation of the dockerfile with a RUN statement, as well as entering the docker container and attempting to install it after the container is already up. The below error statement is from the second attempt.

Is there any easy way to resolve this without having to build it from source?

bash-4.2$ ./bin/logstash-plugin install logstash-filter-greynoise
Validating logstash-filter-greynoise
Installing logstash-filter-greynoise
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "faraday":
  In snapshot (Gemfile.lock):
    faraday (= 0.15.4)

  In Gemfile:
    logstash-filter-greynoise java was resolved to 0.1.5, which depends on
      faraday (~> 0.9.2) java

    logstash-input-elasticsearch java was resolved to 4.5.0, which depends on
      faraday (~> 0.15.4) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

Logstash Version:

bash-4.2$ ./bin/logstash --version
logstash 7.6.0

Java Version:

bash-4.2$ java --version
openjdk 11.0.6 2020-01-14 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.6+10-LTS, mixed mode, sharing)

[fastlorenzo]

Same issue here:

bash-4.2$ /usr/share/logstash/bin/logstash-plugin install logstash-filter-greynoise
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Validating logstash-filter-greynoise
Installing logstash-filter-greynoise
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "faraday":
  In snapshot (Gemfile.lock):
    faraday (= 0.15.4)

  In Gemfile:
    logstash-filter-greynoise java was resolved to 0.1.5, which depends on
      faraday (~> 0.9.2) java

    logstash-input-elasticsearch java was resolved to 4.5.0, which depends on
      faraday (~> 0.15.4) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

Logstash version:

bash-4.2$ /usr/share/logstash/bin/logstash --version
logstash 7.6.2

Java version:

bash-4.2$ java --version
openjdk 11.0.6 2020-01-14 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.6+10-LTS, mixed mode, sharing)

I've also tried building v 1.0.6 of the plugin offline and then importing it in docker, but get similar error:

bash-4.2$ /usr/share/logstash/bin/logstash-plugin install logstash-filter-greynoise-0.1.6.gem
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Validating logstash-filter-greynoise-0.1.6.gem
Installing logstash-filter-greynoise
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "faraday":
  In snapshot (Gemfile.lock):
    faraday (= 0.15.4)

  In Gemfile:
    logstash-filter-greynoise (= 0.1.6) java was resolved to 0.1.6, which depends on
      faraday (= 0.17.1) java

    logstash-input-elasticsearch java was resolved to 4.5.0, which depends on
      faraday (~> 0.15.4) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

Docker version: docker.elastic.co/logstash/logstash:7.6.2

[fastlorenzo]

FYI, installation works if I compile the plugin with faraday v0.15.4:

RUN sed -i 's/= 0.17.1/=0.15.4/' logstash-filter-greynoise.gemspec

ash-4.2$ /usr/share/logstash/bin/logstash-plugin install /tmp/logstash-filter-greynoise-0.1.6.gem 
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Validating /tmp/logstash-filter-greynoise-0.1.6.gem
Installing logstash-filter-greynoise
Installation successful

Not sure if that impacts the plugin though, didn't test it yet.

For those interested here is the Dockerfile used to build it:

FROM docker.elastic.co/logstash/logstash:7.6.2 AS build
USER root
RUN yum install -y git wget
RUN /usr/share/logstash/vendor/jruby/bin/jruby -S gem install bundler
WORKDIR /tmp
RUN git clone https://github.com/nicksherron/logstash-filter-greynoise.git
WORKDIR /tmp/logstash-filter-greynoise
RUN sed -i 's/= 0.17.1/=0.15.4/' logstash-filter-greynoise.gemspec
RUN /usr/share/logstash/vendor/jruby/bin/jruby -S bundle install
RUN /usr/share/logstash/vendor/jruby/bin/jruby -S gem build logstash-filter-greynoise.gemspec

FROM docker.elastic.co/logstash/logstash:7.6.2

COPY --from=build /tmp/logstash-filter-greynoise/logstash-filter-greynoise-0.1.6.gem /tmp/logstash-filter-greynoise-0.1.6.gem
RUN /usr/share/logstash/bin/logstash-plugin install /tmp/logstash-filter-greynoise-0.1.6.gem

[blakejarvis]

Thanks for looking into this, @fastlorenzo, I replaced my Dockerfile with yours and can confirm I'm getting Greynoise data through the logstash pipeline.

Is it worth submitting a pull request for this change within the gemspec file? I would like to ensure I (or others) won't run into future problems from changing the version requirement on the fly with sed.

[nicksherron]

Hey guys, sorry for the delay! This should be fixed now, @blakejarvis or @fastlorenzo would you mind trying again and confirming before I close this?

[blakejarvis]

@nicksherron I can confirm it built successfully. Thanks for the quick updates.

[nicksherron]

@nicksherron I can confirm it built successfully. Thanks for the quick updates.

Good to hear thanks for checking.