MongoDB River not indexing ElasticSearch data

[hackeys]

Hi ,

I am getting below error when tried to consume mongiDB river 2.0.9 with ElasticSearch 1.6.0-1 and MongoDB 3.0.4 on Centos 7.0

tail -f /var/log/elasticsearch/elasticsearch.log [2015-07-16 16:13:14,287][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] Started river montooops [2015-07-16 16:13:14,289][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] Started river montooraw [2015-07-16 16:13:14,808][ERROR][org.elasticsearch.river.mongodb.OplogSlurper] Exception while looping in cursor java.lang.NullPointerException at org.elasticsearch.river.mongodb.OplogSlurper.getObjectIdFromOplogEntry(OplogSlurper.java:444) at org.elasticsearch.river.mongodb.OplogSlurper.processOplogEntry(OplogSlurper.java:228) at org.elasticsearch.river.mongodb.OplogSlurper.run(OplogSlurper.java:109) at java.lang.Thread.run(Thread.java:745) [2015-07-16 16:13:14,811][INFO ][org.elasticsearch.river.mongodb.OplogSlurper] Slurper is stopping. River has status RUNNING [2015-07-16 16:16:09,133][INFO ][river.mongodb.util ] setRiverStatus called with events - RUNNING

My River Config is { "index": { "name": "montooraw", "type": "Montoo" }, "mongodb": { "db": "Montoo", "servers": [ { "port": 27017, "host": "X.X.X.X" } ], "collection": "MontooRaw", "options": { "secondary_read_preference": true } }, "type": "mongodb" }

[twistedfategit]

I think. river-mongodb recomment elasticsearch 1.4.2 and mongodb 3.0.x

MongoDB River Plugin ElasticSearch MongoDB TokuMX master 1.4.2 3.0.0 1.5.1 2.0.9 1.4.2 3.0.0 1.5.1 2.0.5 1.4.2 2.6.6 1.5.1 2.0.2 1.3.5 2.6.5 1.5.1

. Downgrade elasticsearch 1.6 to 1.4.2 .

[akluffy]

Thanks!

[shubhamrajvanshi]

I am getting error while importing data from mongodb. Here are the versions Mongo: 3.0.0 Elasticsearch :1.4.2 River plugin: 2.0.9

it replicates only few data and then stops. Here are the logs.

[2015-07-24 03:41:02,052][INFO ][node ] [elastic node1] version[1.4.2], pid[16738], build[927caff/2014-12-16T14:11:12Z] [2015-07-24 03:41:02,052][INFO ][node ] [elastic node1] initializing ... [2015-07-24 03:41:02,106][INFO ][plugins ] [elastic node1] loaded [mongodb-river, mapper-attachments, cloud-aws], sites [head, river-mongodb] [2015-07-24 03:41:04,964][INFO ][node ] [elastic node1] initialized [2015-07-24 03:41:04,964][INFO ][node ] [elastic node1] starting ... [2015-07-24 03:41:05,027][INFO ][transport ] [elastic node1] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/52.3.164.140:9300]} [2015-07-24 03:41:05,034][INFO ][discovery ] [elastic node1] datarush elastic_cluster/J5aLSk7YTWWn_a0OPGAIjw [2015-07-24 03:41:35,035][WARN ][discovery ] [elastic node1] waited for 30s and no initial state was set by the discovery [2015-07-24 03:41:35,040][INFO ][http ] [elastic node1] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/52.3.164.140:9200]} [2015-07-24 03:41:35,040][INFO ][node ] [elastic node1] started [2015-07-24 03:41:37,430][INFO ][cluster.service ] [elastic node1] new_master [elastic node1][J5aLSk7YTWWn_a0OPGAIjw][ip-172-31-39-187][inet[/52.3.164.140:9300]], reason: zen-disco-join (elected_as_master) [2015-07-24 03:42:08,054][INFO ][cluster.service ] [elastic node1] added {[elastic node2][zFZwPmfuRaeW_nYbAYetew][ip-172-31-39-188][inet[/52.3.172.64:9300]],}, reason: zen-disco-receive(join from node[[elastic node2][zFZwPmfuRaeW_nYbAYetew][ip-172-31-39-188][inet[/52.3.172.64:9300]]]) [2015-07-24 03:42:09,185][INFO ][gateway ] [elastic node1] recovered [2] indices into cluster_state [2015-07-24 03:42:09,301][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] MongoDB River Plugin - version[2.0.9] - hash[73ddea5] - time[2015-04-06T21:16:46Z] [2015-07-24 03:42:09,304][INFO ][river.mongodb.util ] setRiverStatus called with mongodb - RUNNING [2015-07-24 03:42:09,376][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] River mongodb startup pending [2015-07-24 03:42:09,399][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] Starting river mongodb [2015-07-24 03:42:09,405][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] MongoDB options: secondaryreadpreference [false], drop_collection [false], include_collection [], throttlesize [5000], gridfs [false], filter [null], db [partb], collection [house], script [null], indexing to [partb]/[house] [2015-07-24 03:42:09,407][INFO ][river.mongodb ] [elastic node1] Creating MongoClient for [[52.2.127.199:27017]] [2015-07-24 03:42:09,558][INFO ][org.elasticsearch.river.mongodb.MongoConfigProvider] MongoDB version - 3.0.4 [2015-07-24 03:42:09,753][ERROR][org.elasticsearch.river.mongodb.CollectionSlurper] Cannot import collection house into existing index [2015-07-24 03:42:09,754][INFO ][river.mongodb.util ] setRiverStatus called with mongodb - INITIAL_IMPORT_FAILED [2015-07-24 03:42:09,794][INFO ][river.mongodb ] [elastic node1] Creating MongoClient for [[ip-172-31-34-107:27017]] [2015-07-24 03:42:09,812][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] Started river mongodb [2015-07-24 03:45:51,383][INFO ][river.mongodb.util ] setRiverStatus called with mongodb - RUNNING [2015-07-24 03:45:55,227][INFO ][river.mongodb.util ] setRiverStatus called with mongodb1 - RUNNING [2015-07-24 03:48:36,420][INFO ][cluster.service ] [elastic node1] added {[elastic node3][W8Mpif34QsmFK_VDfzgoaQ][ip-172-31-38-176][inet[/52.3.108.108:9300]],}, reason: zen-disco-receive(join from node[[elastic node3][W8Mpif34QsmFK_VDfzgoaQ][ip-172-31-38-176][inet[/52.3.108.108:9300]]]) [2015-07-24 03:57:10,295][INFO ][cluster.metadata ] [elastic node1] [_river] update_mapping mongodb [2015-07-24 03:57:10,335][INFO ][cluster.metadata ] [elastic node1] [_river] update_mapping mongodb1 [2015-07-24 04:21:53,570][INFO ][cluster.metadata ] [elastic node1] [partb] update_mapping house

am i missing something