search

logstash-plugins / logstash-filter-jdbc_streaming

A Logstash filter that can enrich events with data from a database
Apache License 2.0
12 stars 23 forks source link

Multiple JAR files in classpath #21

Closed buinauskas closed 5 years ago

buinauskas commented 5 years ago

I'd like to report a bug that plugin does not recognize multiple JAR files in class path. For instance that's my config file for this filter:

jdbc_streaming {
    id => "via-categories-stream"
    jdbc_user => "user"
    jdbc_driver_library => "/usr/share/logstash/drivers/GoogleBigQueryJDBC42.jar,/usr/share/logstash/drivers/jackson-core-2.1.3.jar,/usr/share/logstash/drivers/google-oauth-client-1.24.1.jar,/usr/share/logstash/drivers/google-http-client-jackson2-1.24.1.jar,/usr/share/logstash/drivers/google-http-client-1.24.1.jar,/usr/share/logstash/drivers/google-api-services-bigquery-v2-rev400-1.24.1.jar,/usr/share/logstash/drivers/google-api-client-1.24.1.jar"
    jdbc_driver_class => "com.simba.googlebigquery.jdbc42.Driver"
    jdbc_connection_string => ".."
    statement => "SELECT categoryId, countryId, brandId, supplierId FROM UNNEST ((SELECT filters FROM migration_search.categories WHERE categoryid=:id))"
    parameters => { "id" => "categoryid"}
    target => "filters"
}

And that's the exception I receive:

[2019-01-22T13:49:41,291][ERROR][logstash.filters.jdbcstreaming] Invalid setting for jdbc_streaming filter plugin:

filter {
  jdbc_streaming {
    # This setting must be a path
    # File does not exist or cannot be opened /usr/share/logstash/drivers/GoogleBigQueryJDBC42.jar,/usr/share/logstash/drivers/jackson-core-2.1.3.jar,/usr/share/logstash/drivers/google-oauth-client-1.24.1.jar,/usr/share/logstash/drivers/google-http-client-jackson2-1.24.1.jar,/usr/share/logstash/drivers/google-http-client-1.24.1.jar,/usr/share/logstash/drivers/google-api-services-bigquery-v2-rev400-1.24.1.jar,/usr/share/logstash/drivers/google-api-client-1.24.1.jar
    jdbc_driver_library => "/usr/share/logstash/drivers/GoogleBigQueryJDBC42.jar,/usr/share/logstash/drivers/jackson-core-2.1.3.jar,/usr/share/logstash/drivers/google-oauth-client-1.24.1.jar,/usr/share/logstash/drivers/google-http-client-jackson2-1.24.1.jar,/usr/share/logstash/drivers/google-http-client-1.24.1.jar,/usr/share/logstash/drivers/google-api-services-bigquery-v2-rev400-1.24.1.jar,/usr/share/logstash/drivers/google-api-client-1.24.1.jar"
    ...
  }
}

Very same configuration works with JDBC input filter, so I assume that it doesn't recognize multiple JARs in classpath

darshanhegde13 commented 5 years ago

@buinauskas: Even I am also facing the same issue, Let me know if you are able to resolve this issue.

guyboertje commented 5 years ago

Until we can fix this (and the driver loading problem on JDK9+) try to put a copy of the jars in <LS instal dir>/logstash-core/lib/jars/ directory.

robbavey commented 5 years ago

I believe this should have been fixed by #35, and released in version 1.0.10