search

richardwilly98 / elasticsearch-river-mongodb

MongoDB River Plugin for ElasticSearch
1.12k stars 215 forks source link

[elasticsearch-river-mongodb] Running into java.lang.ClassNotFoundException: mongodb #96

Closed mongodbfan closed 11 years ago

mongodbfan commented 11 years ago

Hi,

Platform: Mac OS X

As suggested by https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki, i have copied (manually - as none of ./plugin --install steps are working) elasticsearch-river-mongodb-1.6.9.jar & mongo-java-driver-2.11.1.jar to ES_HOME/plugins/river-mongodb. Restarted ElasticSearch (./elasticsearch -f).

Made sure CLASSPATH has ES_HOME/plugins/river-mongodb.

Sample river created: curl -XPUT "localhost:9200/_river/mydb/_meta" -d ' { "type": "mongodb", "mongodb": { "servers": [ { "host": "127.0.0.1", "port": 27017 } ], "options": { "secondary_read_preference": false }, "db": "mydb", "collection": "users" }, "index": { "name": "mydb", "type": "users" } }'

When I start elastic search, I am running into class not found exception. Here is the complete stack. Can someone help? Thanks much.

[2013-06-30 10:55:53,511][INFO ][cluster.metadata ] [Tzabaoth] [_river] update_mapping mydb [2013-06-30 10:55:53,518][WARN ][river ] [Tzabaoth] failed to create river [mongodb][mydb] org.elasticsearch.common.settings.NoClassSettingsException: Failed to load class with value [mongodb] at org.elasticsearch.river.RiverModule.loadTypeModule(RiverModule.java:87) at org.elasticsearch.river.RiverModule.spawnModules(RiverModule.java:58) at org.elasticsearch.common.inject.ModulesBuilder.add(ModulesBuilder.java:44) at org.elasticsearch.river.RiversService.createRiver(RiversService.java:135) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:270) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:264) at org.elasticsearch.action.support.TransportAction$ThreadedActionListener$1.run(TransportAction.java:87) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:680) Caused by: java.lang.ClassNotFoundException: mongodb at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) at org.elasticsearch.river.RiverModule.loadTypeModule(RiverModule.java:73) ... 9 more

mongodbfan commented 11 years ago

more version info for the above issue:

elasticsearch: 0.90.2 mongodb: 2.4.4 rivermongodb plugin jars: 1.6.9 analysis-combo: 1.3.0

richardwilly98 commented 11 years ago

Hi,

Can you please provide ES log file?

Thanks, Richard.

Sent via BlackBerry by AT&T

-----Original Message----- From: mongodbfan notifications@github.com Date: Sun, 30 Jun 2013 13:43:11 To: richardwilly98/elasticsearch-river-mongodbelasticsearch-river-mongodb@noreply.github.com Reply-To: richardwilly98/elasticsearch-river-mongodb reply@reply.github.com Subject: Re: [elasticsearch-river-mongodb] [elasticsearch-river-mongodb] Running into java.lang.ClassNotFoundException: mongodb (#96)

more version info for the above issue:

elasticsearch: 0.90.2 mongodb: 2.4.4 rivermongodb plugin jars: 1.6.9 analysis-combo: 1.3.0

Reply to this email directly or view it on GitHub: https://github.com/richardwilly98/elasticsearch-river-mongodb/issues/96#issuecomment-20255025

mongodbfan commented 11 years ago

Sure. Here is the one.

[2013-06-30 16:03:54,367][INFO ][node ] [Destiny I] {0.90.2}[20442]: initializing ... [2013-06-30 16:03:54,438][INFO ][plugins ] [Destiny I] loaded [mapper-attachments, analysis-combo, analysis-skos, lang-javascript], sites []

[2013-06-30 16:03:56,794][INFO ][node ] [Destiny I] {0.90.2}[20442]: starting ... [2013-06-30 16:03:56,882][INFO ][transport ] [Destiny I] bound_address {inet[/0:0:0:0:0:0:0:0%0:9300]}, publish_address {inet[/192.168.33.131:9300]} [2013-06-30 16:03:59,945][INFO ][cluster.service ] [Destiny I] new_master [Destiny I][EXNtau0xQ8C5TPnCNAFwVw][inet[/192.168.33.131:9300]], reason: zen-disco-join (elected_as_master) [2013-06-30 16:03:59,981][INFO ][discovery ] [Destiny I] elasticsearch/EXNtau0xQ8C5TPnCNAFwVw [2013-06-30 16:03:59,995][INFO ][http ] [Destiny I] bound_address {inet[/0:0:0:0:0:0:0:0%0:9200]}, publish_address {inet[/192.168.33.131:9200]}

[2013-06-30 16:04:00,892][INFO ][gateway ] [Destiny I] recovered [2] indices into cluster_state [2013-06-30 16:04:00,921][WARN ][river ] [Destiny I] failed to create river [mongodb][mydb2] org.elasticsearch.common.settings.NoClassSettingsException: Failed to load class with value [mongodb] at org.elasticsearch.river.RiverModule.loadTypeModule(RiverModule.java:87) at org.elasticsearch.river.RiverModule.spawnModules(RiverModule.java:58) at org.elasticsearch.common.inject.ModulesBuilder.add(ModulesBuilder.java:44) at org.elasticsearch.river.RiversService.createRiver(RiversService.java:135) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:270) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:264) at org.elasticsearch.action.support.TransportAction$ThreadedActionListener$1.run(TransportAction.java:87) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:680) Caused by: java.lang.ClassNotFoundException: mongodb at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) at org.elasticsearch.river.RiverModule.loadTypeModule(RiverModule.java:73) ... 9 more [2013-06-30 16:04:00,938][WARN ][river ] [Destiny I] failed to create river [mongodb][mydb] org.elasticsearch.common.settings.NoClassSettingsException: Failed to load class with value [mongodb] at org.elasticsearch.river.RiverModule.loadTypeModule(RiverModule.java:87) at org.elasticsearch.river.RiverModule.spawnModules(RiverModule.java:58) at org.elasticsearch.common.inject.ModulesBuilder.add(ModulesBuilder.java:44) at org.elasticsearch.river.RiversService.createRiver(RiversService.java:135) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:270) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:264) at org.elasticsearch.action.support.TransportAction$ThreadedActionListener$1.run(TransportAction.java:87) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:680) Caused by: java.lang.ClassNotFoundException: mongodb at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) at org.elasticsearch.river.RiverModule.loadTypeModule(RiverModule.java:73) ... 9 more

richardwilly98 commented 11 years ago

Hi,

From where did you download the jar files?

Can you please try this url [1]?

BTW - you said you could nod install it using plugin command what kind of error do you get? No issues have been reported so on plugin command.

[1] - https://oss.sonatype.org/content/repositories/releases/com/github/richardwilly98/elasticsearch/elasticsearch-river-mongodb/1.6.9/elasticsearch-river-mongodb-1.6.9.zip

Thanks, Richard.

mongodbfan commented 11 years ago

Richard,

Thanks for your time.

Here is the command executed from the console (that is ES's bin/plugin).

bin/plugin -install https://oss.sonatype.org/content/repositories/releases/com/github/richardwilly98/elasticsearch/elasticsearch-river-mongodb/1.6.9/elasticsearch-river-mongodb-1.6.9.zip -> Installing https://oss.sonatype.org/content/repositories/releases/com/github/richardwilly98/elasticsearch/elasticsearch-river-mongodb/1.6.9/elasticsearch-river-mongodb-1.6.9.zip... Trying http://download.elasticsearch.org/https://-oss.sonatype.org.zip... Trying http://search.maven.org/remotecontent?filepath=https://oss.sonatype.org/-oss.sonatype.org.zip... Trying https://oss.sonatype.org/service/local/repositories/releases/content/https://oss.sonatype.org/-oss.sonatype.org.zip... Trying https://github.com/https://zipball/voss.sonatype.org... (assuming site plugin) Failed to install https://oss.sonatype.org/content/repositories/releases/com/github/richardwilly98/elasticsearch/elasticsearch-river-mongodb/1.6.9/elasticsearch-river-mongodb-1.6.9.zip, reason: failed to download out of all possible locations..., use -verbose to get detailed information

I eventually got the package by typing the URL in browser and followed manual steps of installing them under ES_HOME/plugins/river-mongodb. I have also set the CLASSPATH to cover these 2 jar files. Do you think manual copy disturbed the equation? There were talks about analysis-combo fixing the problem. I got latest version of that too. That did not help.

Are you currently using latest versions of ES + MONGODB + RIVER jars in storing, indexing and searching data?

Thanks.

richardwilly98 commented 11 years ago

Hi,

The wiki says:

%ES_HOME%\bin\plugin.bat -url https://oss.sonatype.org/content/repositories/releases/com/github/richardwilly98/elasticsearch/elasticsearch-river-mongodb/1.6.9/elasticsearch-river-mongodb-1.6.9.zip -install river-mongodb

So please use -url switch and not -install

Thanks, Richard.

mongodbfan commented 11 years ago

Sorry for all these iterations. Tried the above command. Here are the results. Plugin is behaving as if it does not anything about -url or --url or -u commandline option.

myMac$ ./plugin -url https://oss.sonatype.org/content/repositories/releases/com/github/richardwilly98/elasticsearch/elasticsearch-river-mongodb/1.6.9/elasticsearch-river-mongodb-1.6.9.zip -install river-mongodb -v Usage: -u, --url [plugin location] : Set exact URL to download the plugin from -i, --install [plugin name] : Downloads and installs listed plugins [*] -r, --remove [plugin name] : Removes listed plugins -l, --list : List installed plugins -v, --verbose : Prints verbose messages -h, --help : Prints this help message

[*] Plugin name could be: elasticsearch/plugin/version for official elasticsearch plugins (download from download.elasticsearch.org) groupId/artifactId/version for community plugins (download from maven central or oss sonatype) username/repository for site plugins (download from github master)

Message: Command [-url] unknown. myMac$ ./plugin --url https://oss.sonatype.org/content/repositories/releases/com/github/richardwilly98/elasticsearch/elasticsearch-river-mongodb/1.6.9/elasticsearch-river-mongodb-1.6.9.zip --install river-mongodb Usage: -u, --url [plugin location] : Set exact URL to download the plugin from -i, --install [plugin name] : Downloads and installs listed plugins [*] -r, --remove [plugin name] : Removes listed plugins -l, --list : List installed plugins -v, --verbose : Prints verbose messages -h, --help : Prints this help message

[*] Plugin name could be: elasticsearch/plugin/version for official elasticsearch plugins (download from download.elasticsearch.org) groupId/artifactId/version for community plugins (download from maven central or oss sonatype) username/repository for site plugins (download from github master)

Message: Command [--url] unknown.

Thanks.

richardwilly98 commented 11 years ago

Hi,

Unfortunately I do not have a Mac to test the install issue. I'd suggest to post on ES user group [1].

Assuming the plugin is installed manually can you please confirm the file name and size in $ES_HOME\plugins\river-mongodb?

Can you also please delete the current river settings and restart ES?

Provide ES log file.

[1] - https://groups.google.com/forum/#!forum/elasticsearch

Thanks, Richard.

mongodbfan commented 11 years ago

Richard,

Don't worry about download and install part. I will report it to the group you suggested.

Here are the sizes of jar files.

-rw-r--r--@ 1 staff 30431 Jun 17 08:05 elasticsearch-river-mongodb-1.6.9.jar -rw-r--r--@ 1 staff 417206 Apr 30 05:50 mongo-java-driver-2.11.1.jar

Without rivers, it works fine. The problem starts when add a river as plugins kick in.

Thanks.

richardwilly98 commented 11 years ago

Hi,

Event with no river settings defined the plugin should be already loaded. Can you confirm that?

Thanks, Richard.

mongodbfan commented 11 years ago

No. That is not the case. I don't see reference to "river" at all.

Here is the console dump:

./elasticsearch -f [2013-07-01 16:44:12,706][INFO ][node ] [Destroyer, The] {0.90.2}[21746]: initializing ... [2013-07-01 16:44:12,790][INFO ][plugins ] [Destroyer, The] loaded [mapper-attachments, analysis-combo, analysis-skos, lang-javascript], sites []

[2013-07-01 16:44:15,142][INFO ][node ] [Destroyer, The] {0.90.2}[21746]: starting ... [2013-07-01 16:44:15,260][INFO ][transport ] [Destroyer, The] bound_address {inet[/0:0:0:0:0:0:0:0%0:9300]}, publish_address {inet[/172.24.1.46:9300]} [2013-07-01 16:44:18,332][INFO ][cluster.service ] [Destroyer, The] new_master [Destroyer, The][IDoSDPwvQLmBF05iVE_04g][inet[/172.24.1.46:9300]], reason: zen-disco-join (elected_as_master) [2013-07-01 16:44:18,365][INFO ][discovery ] [Destroyer, The] elasticsearch/IDoSDPwvQLmBF05iVE_04g [2013-07-01 16:44:18,378][INFO ][http ] [Destroyer, The] bound_address {inet[/0:0:0:0:0:0:0:0%0:9200]}, publish_address {inet[/172.24.1.46:9200]}

[2013-07-01 16:44:18,420][INFO ][gateway ] [Destroyer, The] recovered [0] indices into cluster_state

mongodbfan commented 11 years ago

OK. My mistake. I have dropped the river-mongodb plugins under bin/plugins and it is a level above.

You are right. Without rivers, it loads the plugin properly.

Sorry for misleading you. I misunderstood ES_HOME as the place where elasticsearch binary is. Anyway, I created river-mongodb folder now under plugins folder along with other plugins. Then started ES. It loaded the plugin jars fine.

It ran into some other error. I am thinking this is the matter of getting the right plugin/package. It might be obvious for you. Here is the dump. Thanks for patiently replying.

/elasticsearch -f [2013-07-01 19:00:14,044][INFO ][node ] [Marvel Boy] {0.90.2}[21902]: initializing ... [2013-07-01 19:00:14,108][INFO ][plugins ] [Marvel Boy] loaded [river-mongodb, mapper-attachments, analysis-combo, analysis-skos, lang-javascript], sites []

[2013-07-01 19:00:16,370][INFO ][node ] [Marvel Boy] {0.90.2}[21902]: starting ... [2013-07-01 19:00:16,471][INFO ][transport ] [Marvel Boy] bound_address {inet[/0:0:0:0:0:0:0:0%0:9300]}, publish_address {inet[/172.24.1.46:9300]} [2013-07-01 19:00:19,539][INFO ][cluster.service ] [Marvel Boy] new_master [Marvel Boy][zS_EiQ36QYWEysKfggsxOg][inet[/172.24.1.46:9300]], reason: zen-disco-join (elected_as_master) [2013-07-01 19:00:19,579][INFO ][discovery ] [Marvel Boy] elasticsearch/zS_EiQ36QYWEysKfggsxOg [2013-07-01 19:00:19,598][INFO ][http ] [Marvel Boy] bound_address {inet[/0:0:0:0:0:0:0:0%0:9200]}, publish_address {inet[/172.24.1.46:9200]}

[2013-07-01 19:00:20,323][INFO ][gateway ] [Marvel Boy] recovered [3] indices into cluster_state [2013-07-01 19:00:20,862][DEBUG][river.mongodb ] [Marvel Boy] [mongodb][mydb] Prefix: [[Marvel Boy] [mongodb][mydb] ] - name: [river.mongodb] [2013-07-01 19:00:20,862][DEBUG][river.mongodb ] River settings: [2013-07-01 19:00:20,862][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb] mongoServersSettings: [{port=27017, host=127.0.0.1}] [2013-07-01 19:00:20,863][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb] Server: 127.0.0.1 - 27017 [2013-07-01 19:00:20,864][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb] Using mongodb server(s): host [127.0.0.1], port [27017] [2013-07-01 19:00:20,864][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb] starting mongodb stream. options: secondaryreadpreference [false], throttlesize [500], gridfs [false], filter [], db [mydb], collection [users], script [null], indexing to [mydb]/[users] [2013-07-01 19:00:20,957][WARN ][river ] [Marvel Boy] failed to get _meta from [mongodb]/[mydb] java.lang.NoSuchMethodError: com.mongodb.MongoClientOptions.builder()Lcom/mongodb/MongoClientOptions$Builder; at org.elasticsearch.river.mongodb.MongoDBRiver.getMongoClient(MongoDBRiver.java:559) at org.elasticsearch.river.mongodb.MongoDBRiver.getAdminDb(MongoDBRiver.java:517) at org.elasticsearch.river.mongodb.MongoDBRiver.isMongos(MongoDBRiver.java:495) at org.elasticsearch.river.mongodb.MongoDBRiver.start(MongoDBRiver.java:462) at org.elasticsearch.river.RiversService.createRiver(RiversService.java:146) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:270) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:264) at org.elasticsearch.action.support.TransportAction$ThreadedActionListener$1.run(TransportAction.java:87) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:680) [2013-07-01 19:00:20,957][DEBUG][river.mongodb ] [Marvel Boy] [mongodb][mydb2] Prefix: [[Marvel Boy] [mongodb][mydb2] ] - name: [river.mongodb] [2013-07-01 19:00:20,960][DEBUG][river.mongodb ] River settings: [2013-07-01 19:00:20,961][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb2] mongoServersSettings: [{port=27017, host=127.0.0.1}] [2013-07-01 19:00:20,962][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb2] Server: 127.0.0.1 - 27017 [2013-07-01 19:00:20,962][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb2] Using mongodb server(s): host [127.0.0.1], port [27017] [2013-07-01 19:00:20,962][INFO ][river.mongodb ] [Marvel Boy] [mongodb][mydb2] starting mongodb stream. options: secondaryreadpreference [false], throttlesize [500], gridfs [false], filter [], db [mydb2], collection [users], script [null], indexing to [mydb2]/[users] [2013-07-01 19:00:20,967][WARN ][river ] [Marvel Boy] failed to get _meta from [mongodb]/[mydb2] java.lang.NoSuchMethodError: com.mongodb.MongoClientOptions.builder()Lcom/mongodb/MongoClientOptions$Builder; at org.elasticsearch.river.mongodb.MongoDBRiver.getMongoClient(MongoDBRiver.java:559) at org.elasticsearch.river.mongodb.MongoDBRiver.getAdminDb(MongoDBRiver.java:517) at org.elasticsearch.river.mongodb.MongoDBRiver.isMongos(MongoDBRiver.java:495) at org.elasticsearch.river.mongodb.MongoDBRiver.start(MongoDBRiver.java:462) at org.elasticsearch.river.RiversService.createRiver(RiversService.java:146) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:270) at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:264) at org.elasticsearch.action.support.TransportAction$ThreadedActionListener$1.run(TransportAction.java:87) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:680)

richardwilly98 commented 11 years ago

Hi,

It looks like ES cannot find a method com.mongodb.MongoClientOptions.builder() mongo-java-driver. This method is available since 2.10.0 [1].

Do you multiple version load in ES class path?

As suggested here [2] I would run ES with -verbose switch to see what is loaded.

[1] - http://api.mongodb.org/java/current/com/mongodb/MongoClientOptions.Builder.html [2] - http://stackoverflow.com/questions/13593614/mongodb-java-lang-nosuchfielderror

Thanks, Richard.

mongodbfan commented 11 years ago

Thank you Richard. I could finally bypass this error.   I have another question. I am trying to use percolate feature in elasticsearch and unable to find an example that prints the results to console or something like that. The one in elasticsearch.org is not very clear on the results part. Do you have any pointers?   Thanks a lot for your time.

From: Richard Louapre notifications@github.com To: richardwilly98/elasticsearch-river-mongodb elasticsearch-river-mongodb@noreply.github.com Cc: mongodbfan bhuvanracham@yahoo.com Sent: Tuesday, July 2, 2013 3:02 AM Subject: Re: [elasticsearch-river-mongodb] [elasticsearch-river-mongodb] Running into java.lang.ClassNotFoundException: mongodb (#96)

Hi, It looks like ES cannot find a method com.mongodb.MongoClientOptions.builder() mongo-java-driver. This method is available since 2.10.0 [1]. Do you multiple version load in ES class path? As suggested here [2] I would run ES with -verbose switch to see what is loaded. [1] - http://api.mongodb.org/java/current/com/mongodb/MongoClientOptions.Builder.html [2] - http://stackoverflow.com/questions/13593614/mongodb-java-lang-nosuchfielderror Thanks, Richard. — Reply to this email directly or view it on GitHub.

richardwilly98 commented 11 years ago

Hi,

Sorry I cannot help I have not used percolator feature yet. Try contacting ES user group.

Thanks, Richard.

matanzg commented 11 years ago

Regarding the "Command [--url] unknown." issue, replace the order of the parameters and it'll work, i.e. enter something like: plugin.bat -i river-mongodv -u https://github.com/downloads/richardwilly98/elasticsearch-river-mongodb/elasticsearch-river-mongodb-1.6.5.zip