logstash-plugins / logstash-integration-jdbc

Logstash Integration Plugin for JDBC, including Logstash Input and Filter Plugins
Apache License 2.0
52 stars 55 forks source link

[Feature Request] Support statement_filepath for filter::jdbc_streaming #51

Open tbennett6421 opened 3 years ago

tbennett6421 commented 3 years ago

Brief: Would it be possible to request support for statement_filepath when using filter { jdbc_streaming }? We currently use statement_filepath for input { jdbc }. I tried both statement_filepath, and statement_path since statement_path was hinted to in the docs unofficially.

https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_streaming.html

The statement (or statement_path) setting still holds the SQL statement but to use bind variables you must use the ? character as a placeholder in the exact order found in the prepared_statement_bind_values array. Some technologies may require connection string properties to be set, see MySQL example below.

Version: logstash 7.9.3

Operating System: any [RHEL7, Ubuntu, OSX]

Config File:

filter {
    jdbc_streaming {

        jdbc_driver_library => "/path/to/mysql-connector-java-5.1.34-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://localhost:3306/mydatabase"
        jdbc_user => "me"
        jdbc_password => "secret"
            statement_filepath => "/etc/logstash/resources/sql-statements/enrichment1.sql"
            target => "enrichment_jdbc_streaming"

        } 
}

Sample Data: N/A

Steps to Reproduce: logstash --debug --config.test_and_exit -f ./logstash.conf

codectified commented 9 months ago

three years later but still an issue...?

erin-doyle commented 3 months ago

I also was confused by the reference to statement_path in the docs but then to encounter an "unknown setting statement_path" error.