yougov / mongo-connector

MongoDB data stream pipeline tools by YouGov (adopted from MongoDB)
Apache License 2.0
1.88k stars 479 forks source link

Mongos Shared Clusters not connecting #258

Closed rockerBOO closed 9 years ago

rockerBOO commented 9 years ago

I am working with Vagrant (VirtualBox) with a Docker based shared cluster. https://github.com/jacksoncage/mongo-docker

My issue is when connecting to the mongos router via mongo-connector to seems to have a connection issue. I am not sure if it is related to sharding, my routing, or something else.

$ mongo-connector -m 172.17.8.100:32783 -t localhost:9200 -d elastic_doc_manager
Logging to mongo-connector.log.
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/mongo_connector/util.py", line 85, in wrapped
    func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/mongo_connector/connector.py", line 342, in run
    **self.ssl_kwargs)
  File "/home/rockerboo/.local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 377, in __init__
    raise ConnectionFailure(str(e))
ConnectionFailure: [Errno -2] Name or service not known, [Errno -2] Name or service not known

mongo from the Host Machine

$ mongo --host 172.17.8.100 --port 32783
MongoDB shell version: 2.6.3
connecting to: 172.17.8.100:32783/test
mongos> show dbs
admin     (empty)
config    0.016GB
database  0.094GB
test      0.031GB

Shard Status

mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "version" : 4,
    "minCompatibleVersion" : 4,
    "currentVersion" : 5,
    "clusterId" : ObjectId("555e98e28da3e9ecc5387e9f")
}
  shards:
    {  "_id" : "set1",  "host" : "set1/mongos1r1.mongo.dev.docker:27017,mongos1r2.mongo.dev.docker:27017" }
    {  "_id" : "set2",  "host" : "set2/mongos2r1.mongo.dev.docker:27017,mongos2r2.mongo.dev.docker:27017" }
    {  "_id" : "set3",  "host" : "set3/mongos3r1.mongo.dev.docker:27017,mongos3r2.mongo.dev.docker:27017" }
  databases:
    {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
    {  "_id" : "test",  "partitioned" : false,  "primary" : "set1" }
    {  "_id" : "database",  "partitioned" : true,  "primary" : "set2" }
        database.testData
            shard key: { "_id" : "hashed" }
            chunks:
                set1    2
                set2    2
                set3    2
            { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong("-6148914691236517204") } on : set1 Timestamp(3, 2) 
            { "_id" : NumberLong("-6148914691236517204") } -->> { "_id" : NumberLong("-3074457345618258602") } on : set1 Timestamp(3, 3) 
            { "_id" : NumberLong("-3074457345618258602") } -->> { "_id" : NumberLong(0) } on : set2 Timestamp(3, 4) 
            { "_id" : NumberLong(0) } -->> { "_id" : NumberLong("3074457345618258602") } on : set2 Timestamp(3, 5) 
            { "_id" : NumberLong("3074457345618258602") } -->> { "_id" : NumberLong("6148914691236517204") } on : set3 Timestamp(3, 6) 
            { "_id" : NumberLong("6148914691236517204") } -->> { "_id" : { "$maxKey" : 1 } } on : set3 Timestamp(3, 7) 
llvtt commented 9 years ago

Mongo Connector tries to establish a connection directly to each of the shards. Can you reach the shards directly from a mongo shell on the same machine running mongo-connector?

rockerBOO commented 9 years ago

I currently can't get the box to work properly, so I can't say for sure if the connectivity issues are resolved. Closing for now and will reopen if I can recreate. Thanks.