search

Azure / azure-diagnostics-tools

Plugins and tools for collecting, processing, managing, and visualizing diagnostics data and configuration
98 stars 92 forks source link

Cannot install logstash-input-azureeventhub #196

Closed src386 closed 5 years ago

src386 commented 5 years ago

I can't install the logstash-input-azureeventhub plugin on my system:

# /usr/share/logstash/bin/logstash-plugin install logstash-input-azureeventhub
Validating logstash-input-azureeventhub
Installing logstash-input-azureeventhub
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "jar-dependencies":
  In snapshot (Gemfile.lock):
    jar-dependencies (= 0.4.0)

  In Gemfile:
    logstash-filter-dissect java was resolved to 1.2.0, which depends on
      jar-dependencies java

    logstash-input-azureeventhub java was resolved to 0.9.8, which depends on
      jar-dependencies (~> 0.3.2) java

    logstash-input-beats java was resolved to 6.0.0, which depends on
      jar-dependencies (~> 0.3, >= 0.3.4) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

Bundler could not find compatible versions for gem "logstash-core":
  In snapshot (Gemfile.lock):
    logstash-core (= 7.2.0)

  In Gemfile:
    logstash-core java

    logstash-core-plugin-api java was resolved to 2.1.16, which depends on
      logstash-core (= 7.2.0) java

    logstash-input-syslog java was resolved to 3.4.1, which depends on
      logstash-filter-grok java was resolved to 4.0.4, which depends on
        logstash-core (>= 5.6.0) java

    logstash-input-azureeventhub java was resolved to 0.9.4, which depends on
      logstash-core (>= 1.4.0, < 2.0.0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

I am not a ruby/gem expert but from what I understand, logstash-input-azureeventhub requires an old logstash-core version and other libs require a new one. This explains why gem can not find a compatible version for everyone.

My setup:

karanotts commented 5 years ago

Had the same problem, got around it by removing Gemfile.lock and re-running plugin installer.

src386 commented 5 years ago

@karanotts:

Had the same problem, got around it by removing Gemfile.lock and re-running plugin installer.

Where is the Gemfile.lock file? Do you mean I need to clone the repository and install from source?

karanotts commented 5 years ago

Apologies, should've been more clear: Gemfile.lock in the main logstash dir. Immediately after renaming Gemfile.lock I was able to successfully install the plugin and confirm it working by running bin/logstash -f /etc/logstash/conf.d/ --config.test_and_exit

src386 commented 5 years ago 
# rm -f /usr/share/logstash/Gemfile.lock
# /usr/share/logstash/bin/logstash-plugin install logstash-input-azureeventhub
Validating logstash-input-azureeventhub
Installing logstash-input-azureeventhub
which: no javac in (/sbin:/bin:/usr/sbin:/usr/bin)
Warning: JAVA_HOME environment variable is not set.
      com.microsoft.azure:azure-eventhubs:0.15.0:compile
Installation successful

Thank you!

src386 commented 5 years ago

Well it seems now logstash is broken now:

Jul  2 09:42:57 test0 logstash: [ERROR] 2019-07-02 11:42:57.629 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (LoadError) You have already activated jar-dependencies 0.4.0, but your Gemfile requires jar-dependencies 0.3.12. Since jar-dependencies is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports jar-dependencies as a default gem.
# /usr/share/logstash/bin/logstash-plugin list
Gem::LoadError: You have already activated jar-dependencies 0.4.0, but your Gemfile requires jar-dependencies 0.3.12. Since jar-dependencies is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports jar-dependencies as a default gem.
  check_for_activated_spec! at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler/runtime.rb:319
                      setup at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler/runtime.rb:31
                       each at org/jruby/RubyArray.java:1792
                       each at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/forwardable.rb:229
                        map at org/jruby/RubyEnumerable.java:833
                      setup at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler/runtime.rb:26
                      setup at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler.rb:107
                     setup! at /usr/share/logstash/lib/bootstrap/bundler.rb:62
                    execute at /usr/share/logstash/lib/pluginmanager/list.rb:17
                        run at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67
                    execute at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/subcommand/execution.rb:11
                        run at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67
                        run at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132
                     <main> at /usr/share/logstash/lib/pluginmanager/main.rb:48

Do not know if this problem is on the azureeventhub plugin scope or related to logstash / gem. Gem dependencies are really a pain. I might just give up and try with a function app + input http.

src386 commented 5 years ago

Sorry I should have read this first: https://github.com/Azure/azure-diagnostics-tools#transition-of-logstash-input-azureeventhub-plugin

As of April 25th 2018, Microsoft has transitioned the code and ownership of the logstash-input-azureeventhub plugin

Indeed the plugin is now upstream available in logstash and work perfectly.