jprante / elasticsearch-jdbc

JDBC importer for Elasticsearch
Apache License 2.0
2.84k stars 710 forks source link

Example for mysql not work for me #941

Open b3g00d opened 7 years ago

b3g00d commented 7 years ago

This is example file.sh for sync data from mysql to elasticsearch.

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
bin=${DIR}/../bin
lib=${DIR}/../lib

echo '
{
    "type" : "jdbc",
    "jdbc" : {
        "url" : "jdbc:mysql://localhost:3306/crmtest",
        "user" : "test123",
        "schedule" : "0 0-59 0-23 ? * *",
        "password" : "test1234@123A",
    "sql" : [{"statement": "select id as _id, full_name as full_name  from user"}],
    "index": "crmtest",
    "type": "user",
    "metrics": {
        "enabled": true
    },
    "elasticsearch" : {
             "cluster" : "elasticsearch",
             "host" : "localhost",
             "port" : 9200
        }
    }
}
' | java \
    -cp "${lib}/*" \
    -Dlog4j.configurationFile=${bin}/log4j2.xml \
    org.xbib.tools.Runner \
    org.xbib.tools.JDBCImporter

I already have

commons-cli-1.3.1.jar               jdbc-driver-csv-1.0.0.jar          lucene-misc-5.5.0.jar
compress-lzf-1.0.2.jar              joda-convert-1.2.jar               lucene-queries-5.5.0.jar
derby-10.12.1.1.jar                 joda-time-2.9.4.jar                lucene-queryparser-5.5.0.jar
elasticsearch-2.3.4.jar             jsr166e-1.1.0.jar                  lucene-sandbox-5.5.0.jar
elasticsearch-helper-2.3.4.0.jar    jts-1.13.jar                       lucene-spatial3d-5.5.0.jar
elasticsearch-jdbc-2.3.4.0.jar      log4j-api-2.5.jar                  lucene-spatial-5.5.0.jar
guava-18.0.jar                      log4j-core-2.5.jar                 lucene-suggest-5.5.0.jar
h2-1.4.191.jar                      log4j-slf4j-impl-2.5.jar           mysql-connector-java-5.1.38.jar
HdrHistogram-2.1.6.jar              lucene-analyzers-common-5.5.0.jar  netty-3.10.5.Final.jar
hppc-0.7.1.jar                      lucene-backward-codecs-5.5.0.jar   postgresql-9.4.1208.jre7.jar
hsqldb-2.3.3.jar                    lucene-core-5.5.0.jar              securesm-1.0.jar
jackson-core-2.6.6.jar              lucene-grouping-5.5.0.jar          slf4j-api-1.7.12.jar
jackson-dataformat-cbor-2.6.6.jar   lucene-highlighter-5.5.0.jar       snakeyaml-1.15.jar
jackson-dataformat-smile-2.6.6.jar  lucene-join-5.5.0.jar              spatial4j-0.5.jar
jackson-dataformat-yaml-2.6.6.jar   lucene-memory-5.5.0.jar            t-digest-3.0.jar

mysql-connector-java on lib folder but when I run this script. It couldn't select anything.

 ✘ ⚙ begood@begood  ~/work/elasticsearch-jdbc-2.3.4.0/bin  ./mysql-test.sh 
./mysql-test.sh: 19: ./mysql-test.sh: Bad substitution
[21:41:30,021][INFO ][metrics.source.plain     ][pool-5-thread-1] totalrows = 0, 29 seconds = 29989 ms, 0 = 0.0 bytes bytes, 0.0 bytes = 0 avg size, 0 dps, 0 MB/s
[21:41:30,021][INFO ][metrics.sink.plain       ][pool-5-thread-1] 29 seconds = 29499 ms, submitted = 0, succeeded = 0, failed = 0, 0 = 0.0 bytes bytes, 0.0 bytes = 0 avg size, 0 dps, 0 MB/s
[21:42:00,019][INFO ][metrics.source.plain     ][pool-5-thread-1] totalrows = 0, 59 seconds = 59989 ms, 0 = 0.0 bytes bytes, 0.0 bytes = 0 avg size, 0 dps, 0 MB/s

Please help me fix this! Note: I have data in database

sanaulla123 commented 7 years ago

The elasticsearch port should be 9300 as it uses TransportClient