jprante / elasticsearch-jdbc

JDBC importer for Elasticsearch
Apache License 2.0
2.84k stars 710 forks source link

Getting InterruptedException #713

Open kfypmqqw opened 8 years ago

kfypmqqw commented 8 years ago

Hey,

We are currently testing our solution based on ES 1.7.1 and jdbc-river 1.7.1.0. This is configure of jdbc


DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
bin=${DIR}/../bin
lib=${DIR}/../lib

echo '{
    "type" : "article",
    "jdbc" : {
        "url" : "jdbc:mysql://172.23.140.103:3306/xxxxx",
        "user" : "root",
        "password" : "xxxxx",
        "schedule": "0 0-59 0-23 ? * *",
        "sql" : "select *, id as _id from v_yth_article_search where modify_date >= date_sub(NOW(), interval 50 minute)",
        "elasticsearch" : {
            "cluster" : "trsdev",
            "host" : "172.23.140.114",
            "port" : 9300
        },
       "index": "article",
       "type" : "article"
    }
}' | java \
       -cp "${lib}/*" \
       -Dlog4j.configurationFile=${bin}/log4j2.xml \
       org.xbib.tools.Runner \
       org.xbib.tools.JDBCImporter

This is a piece of logs:

[18:54:47,547][WARN ][importer                 ][main] schedule canceled
[18:54:47,548][WARN ][BulkTransportClient      ][Thread-1835] no client
[18:55:00,001][INFO ][importer.jdbc            ][pool-2-thread-2] index name = article, concrete index name = article
[18:55:00,003][INFO ][importer.jdbc.context.standard][pool-7668-thread-1] found sink class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSink@12c0929d
[18:55:00,004][INFO ][importer.jdbc.context.standard][pool-7668-thread-1] found source class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSource@42419af
[18:55:00,005][INFO ][BaseTransportClient      ][pool-7668-thread-1] creating transport client, java version 1.7.0_76, effective settings {cluster.name=trsdev, host.0=172.23.140.114, port=9300, sniff=false, autodiscover=false, name=importer, client.transport.ignore_cluster_name=false, client.transport.ping_timeout=5s, client.transport.nodes_sampler_interval=5s}
[18:55:00,006][INFO ][org.elasticsearch.plugins][pool-7668-thread-1] [importer] loaded [support-1.7.1.0-b344fa4], sites []
[18:55:00,031][INFO ][BaseTransportClient      ][pool-7668-thread-1] trying to connect to [inet[/172.23.140.114:9300]]
[18:55:00,040][INFO ][BaseTransportClient      ][pool-7668-thread-1] connected to [[trsdev2][Wg8binO-Rze024C9Dy6oJw][localhost.localdomain][inet[/172.23.140.114:9300]]]
[18:56:00,001][INFO ][importer.jdbc            ][pool-2-thread-2] index name = article, concrete index name = article
[18:56:00,003][INFO ][importer.jdbc.context.standard][pool-7669-thread-1] found sink class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSink@14298356
[18:56:00,003][INFO ][importer.jdbc.context.standard][pool-7669-thread-1] found source class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSource@2d242cfa
[18:56:00,004][INFO ][BaseTransportClient      ][pool-7669-thread-1] creating transport client, java version 1.7.0_76, effective settings {cluster.name=trsdev, host.0=172.23.140.114, port=9300, sniff=false, autodiscover=false, name=importer, client.transport.ignore_cluster_name=false, client.transport.ping_timeout=5s, client.transport.nodes_sampler_interval=5s}
[18:56:00,006][INFO ][org.elasticsearch.plugins][pool-7669-thread-1] [importer] loaded [support-1.7.1.0-b344fa4], sites []
[18:56:00,030][INFO ][BaseTransportClient      ][pool-7669-thread-1] trying to connect to [inet[/172.23.140.114:9300]]
[18:56:00,039][INFO ][BaseTransportClient      ][pool-7669-thread-1] connected to [[trsdev2][Wg8binO-Rze024C9Dy6oJw][localhost.localdomain][inet[/172.23.140.114:9300]]]
[18:56:02,528][WARN ][importer                 ][main] schedule canceled
[18:56:02,530][WARN ][BulkTransportClient      ][Thread-12667] no client
[18:56:02,530][ERROR][importer                 ][pool-2-thread-2] null
java.lang.InterruptedException
        at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:400) ~[?:1.7.0_76]
        at java.util.concurrent.FutureTask.get(FutureTask.java:187) ~[?:1.7.0_76]
        at org.xbib.pipeline.simple.SimplePipelineExecutor.waitFor(SimplePipelineExecutor.java:118) ~[elasticsearch-jdbc-1.7.1.0-uberjar.jar:?]
        at org.xbib.pipeline.simple.MetricSimplePipelineExecutor.waitFor(MetricSimplePipelineExecutor.java:51) ~[elasticsearch-jdbc-1.7.1.0-uberjar.jar:?]
        at org.xbib.tools.Importer.execute(Importer.java:291) ~[elasticsearch-jdbc-1.7.1.0-uberjar.jar:?]
        at org.xbib.tools.Importer.run(Importer.java:176) [elasticsearch-jdbc-1.7.1.0-uberjar.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.7.0_76]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [?:1.7.0_76]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_76]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [?:1.7.0_76]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [?:1.7.0_76]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_76]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_76]
        at java.lang.Thread.run(Thread.java:745) [?:1.7.0_76]
jprante commented 8 years ago

This means that JDBC importer has been interrupted while importing.