search

richardwilly98 / elasticsearch-river-mongodb

MongoDB River Plugin for ElasticSearch
1.12k stars 215 forks source link

River does not work with external mongodb replica set #516

Open phongmh307 opened 9 years ago

phongmh307 commented 9 years ago

River works perfectly on my local environment, but when i deployed it to my production environment which using an external mongodb replica set, it seems to unable to connect.

All port are open. I can connect to ES from my mongo server and I also can connect to mongodb from ES server via mongo

This is the TRACE:

[2015-04-15 03:24:53,173][INFO ][river.mongodb            ] [Ocean] Creating MongoClient for [[130.x.x.x:27017, 104.x.x.y:27018, 104.x.x.x:27000]]
[2015-04-15 03:24:53,324][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:24:53,325][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:25:33,327][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:25:33,328][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:26:13,329][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:26:13,330][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:26:53,331][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:26:53,332][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:27:33,332][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:27:33,333][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:28:13,334][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:28:13,334][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:28:53,335][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:28:53,336][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:29:33,337][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:29:33,337][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:30:13,338][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:30:13,339][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:30:53,340][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:30:53,341][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:31:33,342][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:31:33,342][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
m-teslya commented 9 years ago

Had exactly the same problem. After couple days of investigation, I could figure it out (Ubuntu 14.04.1 LTS, MongoDb 2.6.5, river 2.0.2, ES 1.3.5). Maybe, it'll work for you (or save someone's time) too.

  1. connect to your mongo shell
  2. run rs.conf()
  3. Check whether host in rs.conf() output matches one, you use in river connection settings

I just created mongo replica set via rs.initiate(), and it seems by default it uses hostname fqdn. After I updated my replica set hostname and recreated river, I got it working. So, you need host in your river configuration match with host from rs.conf() of mongodb

Hope this help

akluffy commented 9 years ago

It doesn't work for me.