jprante / elasticsearch-jdbc

JDBC importer for Elasticsearch
Apache License 2.0
2.84k stars 709 forks source link

Import from mysql doesn't create index #798

Open tenderwinner opened 8 years ago

tenderwinner commented 8 years ago

Hi,

I'm running following script in order to synchronyse artists table from mysql with artists index in elasticsearch but, after execution nothing appears in index as well as index itself: ` bin=/root/elasticsearch-jdbc-2.2.0.1/bin lib=/root/elasticsearch-jdbc-2.2.0.1/lib

echo '{ "type" : "jdbc", "jdbc" : { "url" : "jdbc:mysql://localhost:3306/databasename", "user" : "user", "password" : "Password", "sql" : "select artistid as _id, artist_name, link, short_artist_name from artists", "index" : "artists", "type" : "artist" } }' | java \ -cp "${lib}/*" \ -Dlog4j.configurationFile=${bin}/log4j2.xml \ org.xbib.tools.Runner \ org.xbib.tools.JDBCImporter`

curl 'http://localhost:9200/_cat/indices?v' returns health status index pri rep docs.count docs.deleted store.size pri.store.size

Please advise what do I do wrong?

Configuration: elasticsearch-jdbc-2.2.0.1 java version "1.8.0_77" elasticsearch 2.3.0 Java(TM) SE Runtime Environment (build 1.8.0_77-b03) Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode) mysql Ver 14.14 Distrib 5.6.28, for debian-linux-gnu (x86_64) using EditLine wrapper

tenderwinner commented 8 years ago

Tried with elasticsearch 2.2.0, the same

tenderwinner commented 8 years ago

Problem solved by changing cluster name in elasticsearch config to "elasticsearch". Please advise where does importer take cluster name?

dengshilong commented 8 years ago

You can add cluster name like this

{
    "type" : "jdbc",
    "jdbc" : {
        "url" : "jdbc:mysql://localhost:3306/blog",
        "statefile" : "statefile.json",
        "schedule" : "0 0-59 0-23 ? * *",
        "user" : "blog",
        "password" : "12345678",
        "sql" : [{
                "statement": "select id as _id, post_title as title, post_content as content from wp_posts where post_status = ? and post_modified > ? ",
                "parameter": ["publish", "$metrics.lastexecutionstart"]}
            ],
        "index" : "customer",
        "type" : "blog",
        "metrics": {
            "enabled" : true
        },
        "elasticsearch" : {
             "cluster" : "elasticsearch",
             "host" : "localhost",
             "port" : 9300 
        }   
    }
}
nathan-zhu commented 8 years ago

@dengshilong "sql" : [{ "statement": "select id as _id, post_title as title, post_content as content from wp_posts where post_status = ? and post_modified > ? ", "parameter": ["publish", "$metrics.lastexecutionstart"]} ], how to input content for this "post_status = ? and post_modified > ?", like below? my_scripts.sh $status $post_modified

please let me know thanks.

dengshilong commented 8 years ago

The content for "post_status = ? and post_modified > ?" is from "parameter": ["publish", "$metrics.lastexecutionstart"]}, just run my_scripts.sh is enough.