metrics.lastexecutionend and metrics.lastexecutionstart updated between sql statements

[tl24]

The metrics.lastexecutionend and lastexecutionstart are updated between sql statements in the same run. Thus if you have multiple sql statements, then only the first uses the correct date range, the subsequent statement uses the time from the previous statement's run.

Here's an example file: { "type" : "jdbc", "jdbc" : { "ignore_null_values" : "true", "index_settings" : { "bulk" : { "red" : { "refresh_interval" : { "start" : "-1", "stop" : "30s" } } } }, "index" : "myindex", "sql" : [ { "statement" : "/shared/indexsql/company.sql", "parameter" : [ "$metrics.lastexecutionstart" ] }, { "statement" : "/shared/indexsql/location.sql", "parameter" : [ "$metrics.lastexecutionstart" ] }, { "statement" : "/shared/indexsql/quotablelocation.sql", "parameter" : [ "$metrics.lastexecutionstart" ] }, { "statement" : "/shared/indexsql/contact.sql", "parameter" : [ "$metrics.lastexecutionstart" ] } ], "statefile" : "/shared/import_data_state.json", "user" : "user", "password" : "secret", "url" : "jdbc:sqlserver://localhost\SQLEXPRESS;databaseName=MyDatabase;applicationName=elasticsearch", "elasticsearch" : { "port" : "9300", "cluster" : "mycluster", "host" : "localhost" } } }

The "company.sql" runs correctly, since its first. The "location.sql" gets the metrics from the "company.sql" run, instead of preserving the values that were in effect when the script started.

[Antoniossss]

Why is this not even commented? I ran into the same problem, any changes of hotfixing this?

[Antoniossss]

@tl24 Iv created a pull request with fix for this issue. I have struggled with it myself as well. If it is not merged yet, feel froo to checkout https://github.com/Antoniossss/elasticsearch-jdbc/tree/issue-683 and build importer yourself (I already did this) as a temporary solution.