How to select incremental data from a table

[chsnake]

I want to select incremental data from a table to elasticsearch. In my case,i use mysql and create a field "date_time" which's type is datetime(yyyy-MM-dd HH:mm:ss) in a table . below is my import scripts.

@echo off

set DIR=%~dp0 set LIB="%DIR%..\lib*" set BIN="%DIR%\"

REM ??? echo {^ "type" : "jdbc",^ "jdbc" : {^ "url" : "jdbc:mysql://localhost:3306/ainewsdb",^ "user" : "root",^ "password" : "123456",^ "statefile" : "statefile.json",^ "sql" : [^ {^ "statement" : "select Id as _id ,original_text,nlp_keywords,query_keywords,ip, date_time,total_num from ai_news_query where date_time > ?",^ "parameter" : [ "$metrics.lastexecutionstart" ]^ }^ ],^ "treat_binary_as_string" : true,^ "elasticsearch" : {^ "cluster" : "elasticSearch",^ "host" : "localhost",^ "port" : 9300^ },^ "index" : "test",^ "type" : "query_test",^ "schedule" : "0 0-59 0-23 ? * *"^ }^ }^ | "%JAVA_HOME%\bin\java" -cp "%LIB%" -Dlog4j.configurationFile="%BIN%\log4j2.xml" "org.xbib.tools.Runner" "org.xbib.tools.JDBCImporter"

1.when i create a new record which date_time is much later than current time(eg:2016-05-13 11:21:00), the record is not import into elasticsearch; 2.there is nothing available information in the jdbc.log. how to do it?

[dengshilong]

1. You should use UTC time
2. In log4j2.xml, change log level to debug

[chsnake]

l have solved the problem, the data type should be datestamp in mysql. Thanks.