jprante / elasticsearch-jdbc

JDBC importer for Elasticsearch
Apache License 2.0
2.84k stars 710 forks source link

Incremental Load - lastexecutionend #663

Open sandesh2014 opened 8 years ago

sandesh2014 commented 8 years ago

Hello @jprante

I'm facing the issue while using incremental load using 'lastexecutionend' parameter. Hereby furnishing feeder script and log. Could you pl. help me out.

!/bin/sh

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" export ES_HOME=/usr/share/elasticsearch bin=/opt/elasticsearch-jdbc-1.7.2.1/bin lib=/opt/elasticsearch-jdbc-1.7.2.1/lib

echo ' {

"type" : "jdbc",
"jdbc" : {
    "metrics" : {
        "enabled" : true,
        "counter" : 0
    },
    "url" : "jdbc:sqlserver://localhost:1433;databaseName=test",
    "user" : "test",
    "password" : "test",
             "sql" : [
        {
            "statement" : "SELECT ID as _id, name FROM test where last_modified > ?",
            "parameter" : [ "$lastexecutionend" ]
        }
    ],
    "statefile" : "statefile.json",
    "index" : "index1",
    "type" : "index1",
    "elasticsearch" : {
    "cluster" : "els101",
    "host" : "localhost",
    "port" : 9300
}

} } ' | java \ -cp "/opt/elasticsearch-jdbc-1.7.2.1/lib/*" \ -Dlog4j.configurationFile=${bin}/log4j2.xml \ org.xbib.tools.Runner \ org.xbib.tools.JDBCImporter

Feeder Logs: [ERROR][importer.jdbc.context.standard][pool-2-thread-1] com.microsoft.sqlserver.jdbc.SQLServerException: Conversion failed when converting date and/or time from character string. java.io.IOException: com.microsoft.sqlserver.jdbc.SQLServerException: Conversion failed when converting date and/or time from character string. at org.xbib.elasticsearch.jdbc.strategy.standard.StandardSource.fetch(StandardSource.java:639) ~[elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at org.xbib.elasticsearch.jdbc.strategy.standard.StandardContext.fetch(StandardContext.java:215) [elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at org.xbib.elasticsearch.jdbc.strategy.standard.StandardContext.execute(StandardContext.java:190) [elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at org.xbib.tools.JDBCImporter.process(JDBCImporter.java:118) [elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at org.xbib.tools.Importer.newRequest(Importer.java:241) [elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at org.xbib.tools.Importer.newRequest(Importer.java:57) [elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at org.xbib.pipeline.AbstractPipeline.call(AbstractPipeline.java:86) [elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at org.xbib.pipeline.AbstractPipeline.call(AbstractPipeline.java:17) [elasticsearch-jdbc-1.7.2.1-uberjar.jar:?] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [?:1.7.0_71] Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Conversion failed when converting date and/or time from character string. at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216) ~[sqljdbc4.jar:?] at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:254) ~[sqljdbc4.jar:?] at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:84) ~[sqljdbc4.jar:?] at com.microsoft.sqlserver.jdbc.SQLServerResultSet.(SQLServerResultSet.java:311) ~[sqljdbc4.jar:?] at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1526) ~[sqljdbc4.jar:?]

jprante commented 8 years ago

There is no parameter "$lastexecutionend, maybe you want $metrics.lastexecutionend.