logstash-plugins / logstash-input-jdbc

Logstash Plugin for JDBC Inputs
Apache License 2.0
449 stars 187 forks source link

Paging bug: JDBC_PAGING_ENABLED and JDBC_PAGE_SIZE degraded after version 4.3.14 #360

Closed bmax closed 4 years ago

bmax commented 4 years ago

Hello! It seems like at the same time prepared statements were introduced in version 4.3.16 is the same time paging functionality broke. I've tested this by rolling back to 4.3.14 and ensuring that jdbc_paging_enabled works. I am also working on a PR for a fix but in the mean time I thought I'd get this issue out first in case anyone else is having trouble.

The problem resides here: https://github.com/logstash-plugins/logstash-input-jdbc/blob/master/lib/logstash/plugin_mixins/jdbc/statement_handler.rb#L34

The variable @jdbc_paging_enabled is not available in that context therefore this will never evaluate as true.

Debugging information

output { elasticsearch { document_id => "visit-%{user_id}-%{merchant_id}" hosts => [ "${ELASTICSEARCH_HOST}" ] index => "program_users" } }


- Steps to Reproduce: Include `jdbc_paging_enabled` and ensure that the select query does not include limit or offset.