logstash-plugins / logstash-filter-jdbc_streaming

A Logstash filter that can enrich events with data from a database
Apache License 2.0
12 stars 23 forks source link

Issues running with MSSQL and JRE11 #25

Closed novaksam closed 5 years ago

novaksam commented 5 years ago

I'm having issues getting it running with JRE11 and MSSQL JDBC.

Here are the errors:

[2019-04-11T10:12:52,426][ERROR][logstash.javapipeline    ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<Sequel::AdapterNotFound: com.microsoft.sqlserver.jdbc.SQLServerDriver not loaded>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.19.0/lib/sequel/adapters/jdbc.rb:56:in `load_driver'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-jdbc_streaming-1.0.5/lib/logstash/plugin_mixins/jdbc_streaming.rb:52:in `prepare_jdbc_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-jdbc_streaming-1.0.5/lib/logstash/filters/jdbc_streaming.rb:194:in `prepare_connected_jdbc_cache'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-jdbc_streaming-1.0.5/lib/logstash/filters/jdbc_streaming.rb:116:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:192:in `block in register_plugins'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:447:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:204:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:146:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:105:in `block in start'"], :thread=>"#<Thread:0x612bbead@/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:102 run>"}
[2019-04-11T10:12:52,428][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}

And here is my config:

        jdbc_streaming {
          jdbc_validate_connection => true
          jdbc_validation_timeout => 300
          cache_expiration => 60
          cache_size => 20000
          jdbc_driver_library => '/etc/logstash/sqljdbc/mssql-jdbc-7.2.1.jre11.jar'
          jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
          jdbc_connection_string => "jdbc:sqlserver://[Connection Removed]"
          jdbc_user => '[User Removed]'
          jdbc_password => "[Password Removed]"
          statement => "[Query removed]"
          parameters => { "IPAddy" => "queryip" }
          target => "target"

Note that is does work with jre8 and the mssql-jdbc-7.2.1.jre8.jar JAR.

guyboertje commented 5 years ago

Same problem on jdbc_static filter and the jdbc input.

I explain the problem here.

I give a work around here.

We are working on a proper solution.

robbavey commented 5 years ago

This should be fixed by #35, and released in plugin version 1.0.10