jprante / elasticsearch-jdbc

JDBC importer for Elasticsearch
Apache License 2.0
2.84k stars 710 forks source link

elaticsearch 2.1, 'Lucene50' does not exist error #706

Open volkansenturk2012 opened 8 years ago

volkansenturk2012 commented 8 years ago

Dear Jprante, I use ES 2.1, elasticsearch-jdbc-2.0.0.1, java version "1.9.0-ea"

When i import data from mysql, importing data was failed. below are the logs:

[11:04:42,919][INFO ][importer.jdbc ][main] index name = members, concrete index name = members [11:04:42,927][INFO ][importer.jdbc ][pool-3-thread-1] strategy standard: settings = {elasticsearch.cluster=elasticsearch, elasticsearch.host=127.0.0.1, elasticsearch.port=9300, index=members, max_bulk_actions=20000, max_concurrent_bulk_requests=8, password=asdf, sql=sqls/members.sql, treat_binary_as_string=true, url=jdbc:mysql://localhost:3306/tasitcom, user=root}, context = org.xbib.elasticsearch.jdbc.strategy.standard.StandardContext@75d8952a [11:04:42,930][INFO ][importer.jdbc.context.standard][pool-3-thread-1] found sink class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSink@6fc74d6b [11:04:42,935][INFO ][importer.jdbc.context.standard][pool-3-thread-1] found source class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSource@1a8f6 [11:04:42,959][INFO ][org.xbib.elasticsearch.support.client.BaseTransportClient][pool-3-thread-1] creating transport client on Linux Java HotSpot(TM) 64-Bit Server VM Oracle Corporation 1.9.0-ea-b91 1.9.0-ea-b91 with effective settings {autodiscover=false, client.transport.ignore_cluster_name=false, client.transport.nodes_sampler_interval=5s, client.transport.ping_timeout=5s, cluster.name=elasticsearch, host.0=127.0.0.1, name=importer, port=9300, sniff=false} [11:04:42,972][INFO ][org.elasticsearch.plugins][pool-3-thread-1] [importer] loaded [], sites [] [11:04:43,237][INFO ][org.xbib.elasticsearch.support.client.BaseTransportClient][pool-3-thread-1] trying to connect to [127.0.0.1:9300] [11:04:43,324][INFO ][org.elasticsearch.client.transport][pool-3-thread-1] [importer] failed to get node info for {#transport#-1}{127.0.0.1}{127.0.0.1:9300}, disconnecting... org.elasticsearch.transport.RemoteTransportException: [Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.liveness.LivenessResponse]] Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.liveness.LivenessResponse] at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:179) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:138) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) [?:?] at java.lang.Thread.run(Thread.java:747) [?:?] Caused by: java.lang.ExceptionInInitializerError at org.elasticsearch.Version.fromId(Version.java:500) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.Version.readVersion(Version.java:276) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.cluster.node.DiscoveryNode.readFrom(DiscoveryNode.java:326) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.cluster.node.DiscoveryNode.readNode(DiscoveryNode.java:309) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.action.admin.cluster.node.liveness.LivenessResponse.readFrom(LivenessResponse.java:52) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:177) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:138) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) ~[?:?] at java.lang.Thread.run(Thread.java:747) ~[?:?] Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.PostingsFormat with name 'Lucene50' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [es090, completion090, XBloomFilter] at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:109) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.common.lucene.Lucene.(Lucene.java:103) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.Version.fromId(Version.java:500) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.Version.readVersion(Version.java:276) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.cluster.node.DiscoveryNode.readFrom(DiscoveryNode.java:326) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.cluster.node.DiscoveryNode.readNode(DiscoveryNode.java:309) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.action.admin.cluster.node.liveness.LivenessResponse.readFrom(LivenessResponse.java:52) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:177) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:138) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) ~[?:?] at java.lang.Thread.run(Thread.java:747) ~[?:?] [11:04:43,331][ERROR][importer.jdbc.sink.standard][pool-3-thread-1] ingest not properly build, shutting down ingest org.elasticsearch.client.transport.NoNodeAvailableException: no cluster nodes available, check settings {autodiscover=false, client.transport.ignore_cluster_name=false, client.transport.nodes_sampler_interval=5s, client.transport.ping_timeout=5s, cluster.name=elasticsearch, host.0=127.0.0.1, name=importer, port=9300, sniff=false} at org.xbib.elasticsearch.support.client.BaseTransportClient.createClient(BaseTransportClient.java:59) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.elasticsearch.support.client.BaseIngestTransportClient.init(BaseIngestTransportClient.java:27) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.elasticsearch.support.client.transport.BulkTransportClient.init(BulkTransportClient.java:85) ~[elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.elasticsearch.jdbc.strategy.standard.StandardSink$1.create(StandardSink.java:375) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.elasticsearch.jdbc.strategy.standard.StandardSink.setContext(StandardSink.java:85) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.elasticsearch.jdbc.strategy.standard.StandardSink.setContext(StandardSink.java:48) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.elasticsearch.jdbc.strategy.standard.StandardContext.beforeFetch(StandardContext.java:178) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.elasticsearch.jdbc.strategy.standard.StandardContext.execute(StandardContext.java:162) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.tools.JDBCImporter.process(JDBCImporter.java:116) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.tools.Importer.newRequest(Importer.java:215) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.tools.Importer.newRequest(Importer.java:54) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.pipeline.AbstractPipeline.call(AbstractPipeline.java:50) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at org.xbib.pipeline.AbstractPipeline.call(AbstractPipeline.java:16) [elasticsearch-jdbc-2.0.0.1-uberjar.jar:?] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) [?:?] at java.lang.Thread.run(Thread.java:747) [?:?] [11:04:43,347][WARN ][importer.jdbc.sink.standard][pool-3-thread-1] no ingest found [11:08:09,746][WARN ][org.xbib.elasticsearch.support.client.transport.BulkTransportClient][Thread-1] no client

jprante commented 8 years ago

JDBC importer 2.0.0.x is for ES 2.0.0 only.

I have to perform a JDBC importer 2.1.0.x release for ES 2.1.0

jprante commented 8 years ago

Also note, while I very welcome any Java 9 use for development and testing, JDBC importer might not run stable with Java 1.9.0-ea-b91

jprante commented 8 years ago

JDBC importer 2.1.0.0 just released.

volkansenturk2012 commented 8 years ago

thanks a lot jprante