jprante / elasticsearch-jdbc

JDBC importer for Elasticsearch
Apache License 2.0
2.84k stars 710 forks source link

No support of elastic search river with informix #427

Open ashishcoolbhargava opened 9 years ago

ashishcoolbhargava commented 9 years ago

When i try to connect the elastic search jdbc river plugin with postgres or h2 db to get the data into the elastic search engine, it behaves properly.

But in case of informix it always give this kind of error :-

java.sql.SQLException: No suitable driver found for jdbc:informix-sqli:

even after i put the jar file into the plugin/jdbc folder.

anybody has any idea on that.

jprante commented 9 years ago

The information you gave is not sufficient, what version of Informix driver is this?

Please try the latest Version 4.10.JC4DE from http://www14.software.ibm.com/webapp/download/search.jsp?go=y&rs=ifxjdbc

ashishcoolbhargava commented 9 years ago

i using the the latest 4.10.JC4DE drivers from the informix which i have downloaded and copied from the lib folder of the project to the plugin folder of the elastic search engine.

the elastic search jdbc river version i m using is :- elasticsearch-river-jdbc-1.4.0.5

jprante commented 9 years ago

Can you check if there is a file META-INF/services/java.sql.Driver in the driver jar? The content must be a class name for the JDBC driver. If not you can set the system property -Djdbc.drivers=className

See also http://docs.oracle.com/javase/7/docs/api/java/sql/DriverManager.html

ashishcoolbhargava commented 9 years ago

i am not able to see the META-INF/services/java.sql.Driver in the driver jar. so the solution which you are suggesting is in the elastic search engine config properties? please correct if m wrong.. if not where and how i can change this property

ashishcoolbhargava commented 9 years ago

Thanks for the solution. Now i am not getting Driver related issue. But in my informix connection i am trying to hit the database which is located on different server.

so my url looks like :- { "type": "jdbc", "jdbc": { "strategy": "simple", "driver": "com.informix.jdbc.IfxDriver", "url": "jdbc:informix-sqli://cin-jxperfsuse.jenzabar.net:1001/perfcx:informixserver=istarcars", "user": "jxadmin", "password": "jxadmin", "index": "constituent", "type": "user", "sql": "select id as _id, id as constituentid, fullname as detailedname , lastname as lastName , firstname as firstName , middlename as middleName , city as city , st as state , zip as postalCode , ctry as country , ss_no as ssn , phone as telephoneNumber , mail as emailAddress , phone_ext as areaCode from INFORMIX.id_rec" } }

so in this case while my password is "jxadmin" but from the server, i am getting error like :-

[2014-12-30 15:50:46,949][ERROR][river.jdbc.SimpleRiverSource] while opening read connection: jdbc:informix-sqli://cin-jxperfsuse.jenzabar.net:1001/pe rfcx:informixserver=istarcars Incorrect password or user com.informix.asf.IfxASFRemoteException: jxadmin@wlw7-ashishb.jenzabar.net is not known on the database server. java.sql.SQLException: Incorrect password or user com.informix.asf.IfxASFRemoteException: jxadmin@wlw7-ashishb.jenzabar.net is not known on the databa se server. at com.informix.jdbc.IfxSqliConnect.(IfxSqliConnect.java:1144) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at com.informix.jdbc.IfxDriver.connect(IfxDriver.java:254) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:187) at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverSource.getConnectionForReading(SimpleRiverSource.java:217) at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverSource.execute(SimpleRiverSource.java:376) at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverSource.fetch(SimpleRiverSource.java:320) at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverFlow.fetch(SimpleRiverFlow.java:220) at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverFlow.execute(SimpleRiverFlow.java:149) at org.xbib.elasticsearch.plugin.jdbc.RiverPipeline.request(RiverPipeline.java:88) at org.xbib.elasticsearch.plugin.jdbc.RiverPipeline.call(RiverPipeline.java:66) at org.xbib.elasticsearch.plugin.jdbc.RiverPipeline.call(RiverPipeline.java:30) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: com.informix.asf.IfxASFRemoteException: jxadmin@wlw7-ashishb.jenzabar.net at com.informix.asf.Connection.recvConnectionResponse(Connection.java:687) at com.informix.asf.Connection.establishConnection(Connection.java:1618) at com.informix.asf.Connection.(Connection.java:346) at com.informix.jdbc.IfxSqliConnect.(IfxSqliConnect.java:1004) ... 19 more

is it appending my machine with the username.

jprante commented 9 years ago

You should consult the Informix manual, this seems driver related. Most probably, the remote access requires a privilege setting with user and domain name in Informix.

ashishcoolbhargava commented 9 years ago

Thank you so much for the help.

with that can you help me with one more thing.

when i am using H2 database and wanted to maintain the _id from the primary key , in that case the _id which is getting created is auto generated. But the same thing when i use with postgres it works very properly.

the Json request looks like :-

POST data: { "type": "jdbc", "jdbc": { "driver": "org.h2.Driver", "url": "jdbc:h2:tcp://localhost:9092/cx-h2-admissions", "user": "jxadmin", "password": "jxadmin", "index": "constituent", "type": "user", "sql": "select id as _id, id, fullname , lastname , firstname , middlename , city , st, zip , ctry , ss_no , phone , mail , phone_ext as areaCode from INFORMIX.id_rec", "schedule" : "0 0-59 0-23 ? * *" } }

The response which i got from the server when i do a get request is like :-

{ "took" : 41, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 70, "max_score" : 1.0, "hits" : [ { "_index" : "constituent", "_type" : "user", "_id" : "AUqa42PqOuPhBMS3Wvuh", "_score" : 1.0, "_source":{"_ID":2,"ID":2,"FULLNAME":"Cars, Coordinator One","LASTNAME":"Cars","FIRSTNAME":"Coordinator","MIDDLENAME":"One","CITY":"Cincinnati","ST":"OH","ZIP":"45241","CTRY":"USA","SS_NO":"137-79-4432","PHONE":"333-333-4444","MAIL":"","AREACODE":""} },

So here you can see it is not taking the _id value from the id column, could u suggest something in this?

jprante commented 9 years ago

Check how you can get lowercased column names from Informix. In other databases, you can use something like

select id as `_id`