TxMongo continuously loses connection to MongoDB

[E4gle]

When connecting to a Atlas managed cluster TxMongo continuously loses connection and the application fails to run smoothly.

What am I missing in order for it to work smoothly ?

2016-12-26T17:52:17+0200 [-] Loading server_test.py...
2016-12-26T17:52:17+0200 [txmongo.connection._Connection#info] Starting factory <txmongo.connection._Connection instance at 0x7f9991fecd40>
2016-12-26T17:52:17+0200 [-] Loaded.
2016-12-26T17:52:17+0200 [twisted.scripts._twistd_unix.UnixAppLogger#info] twistd 16.6.0 starting up.
2016-12-26T17:52:17+0200 [twisted.scripts._twistd_unix.UnixAppLogger#info] reactor class: twisted.internet.epollreactor.EPollReactor.
2016-12-26T17:52:17+0200 [-] WebMongo starting on 8888
2016-12-26T17:52:17+0200 [__builtin__.WebMongo#info] Starting factory <__builtin__.WebMongo instance at 0x7f9991feccf8>
2016-12-26T17:52:17+0200 [MongoProtocol,client] 'TxMongo lost connection to MongoDB.'
2016-12-26T17:52:17+0200 [MongoProtocol,client] 'TxMongo lost connection to MongoDB.'
2016-12-26T17:52:18+0200 [MongoProtocol,client] 'TxMongo lost connection to MongoDB.'
2016-12-26T17:52:18+0200 [MongoProtocol,client] 'TxMongo lost connection to MongoDB.'
2016-12-26T17:52:18+0200 [MongoProtocol,client] <twisted.internet.tcp.Connector instance at 0x7f9991fece18> will retry in 2 seconds
2016-12-26T17:52:18+0200 [txmongo.connection._Connection#info] Stopping factory <txmongo.connection._Connection instance at 0x7f9991fecd40>
2016-12-26T17:52:21+0200 [txmongo.connection._Connection#info] Starting factory <txmongo.connection._Connection instance at 0x7f9991fecd40>
2016-12-26T17:52:21+0200 [MongoProtocol,client] 'TxMongo lost connection to MongoDB.'
2016-12-26T17:52:21+0200 [MongoProtocol,client] 'TxMongo lost connection to MongoDB.'
2016-12-26T17:52:22+0200 [MongoProtocol,client] 'TxMongo lost connection to MongoDB.'
2016-12-26T17:52:22+0200 [MongoProtocol,client] <twisted.internet.tcp.Connector instance at 0x7f9991fece18> will retry in 3 seconds
2016-12-26T17:52:22+0200 [txmongo.connection._Connection#info] Stopping factory <txmongo.connection._Connection instance at 0x7f9991fecd40>
^C2016-12-26T17:52:22+0200 [-] Received SIGINT, shutting down.
2016-12-26T17:52:22+0200 [-] (TCP Port 8888 Closed)
2016-12-26T17:52:22+0200 [__builtin__.WebMongo#info] Stopping factory <__builtin__.WebMongo instance at 0x7f9991feccf8>
2016-12-26T17:52:22+0200 [-] Main loop terminated.
2016-12-26T17:52:22+0200 [twisted.scripts._twistd_unix.UnixAppLogger#info] Server Shut Down.

#!/usr/bin/env python
#
# requires cyclone:
#  http://github.com/fiorix/cyclone
# run:
#  twistd -ny cyclone_server.tac

import cyclone.web
from twisted.application import service, internet
from txmongo.connection import ConnectionPool
from twisted.internet import defer
import txmongo

class IndexHandler(cyclone.web.RequestHandler):
    @defer.inlineCallbacks
    def get(self):

        name = self.get_argument("name")
        try:
            result = yield self.settings.db.find({"name": name})
        except Exception, e:
            self.write("find failed: %s\n" % str(e))
        else:
            self.write("result(s): %s\n" % repr(result))
        self.finish()

class WebMongo(cyclone.web.Application):
    def __init__(self):
        handlers = [
            (r"/", IndexHandler),
        ]

        uri = "mongodb://USER:PASSWORD@HOST1:PORT,HOST2:PORT,HOST3:PORT/DB?ssl=true&replicaSet=REPLICA&authSource=admin"
        mongo = ConnectionPool(uri, pool_size=1)

        settings = {
            "db": mongo.test
        }

        cyclone.web.Application.__init__(self, handlers, **settings)

application = service.Application("webmongo")
srv = internet.TCPServer(8888, WebMongo(), interface="127.0.0.1")
srv.setServiceParent(application)

[IlyaSkriblovsky]

Are you sure all HOST[123]:PORT addresses are accessible from your host? TxMongo lost connection to MongoDB is usually logged when txmongo fails to connect to one of the servers.

Unfortunately, it is hard to reproduce without access to Atlas managed cluster.

[E4gle]

The cluster URI is valid:

• pymongo driver successfully connects to the same URI
• MongoDB Compass tool successfully connects to the same URI.

I realize this is difficult to reproduce, is there anything I can provide to help understand the cause of this issue ? (Logs or the URI in private or anything else to help solve this)

[IlyaSkriblovsky]

@E4gle Is txmongo able to execute any query? Or all queries fail/timed out?

[E4gle]

The errors begin as soon as the server application starts as seen in the logs posted above I have not seen it perform any query successfully, I am beginning to suspect that the txmongo driver does not authenticate with the cluster successfully.

[E4gle]

The Atlas documentation state that "Atlas uses TLS/SSL to encrypt the connections to your databases." https://docs.atlas.mongodb.com/setup-cluster-security/#security-features-and-setup

Does txmongo establish a SSL connection if it is set in the URI regardless of whether an ssl_context_factory is provided ?

[IlyaSkriblovsky]

@E4gle I've noticed ssl=true in your URL, but txmongo doesn't handle this option, at least for now. You should provide ssl_context_factory parameter to ConnectionPool constructor. Please see https://github.com/twisted/txmongo/blob/master/tests/test_auth.py#L472 for example of how to create one.

[E4gle]

Atlas is a managed cluster so I do not have access to the shell so I cannot create the certificate and key for the ssl_context_factory

Is there any way to create them without having access to the shell ?

[IlyaSkriblovsky]

@E4gle Sorry, example I've linked is overcomplicated because it uses X509 authentication. Since you're using username/password, you probably can just use ssl.ClientContextFactory() as ssl_context_factory. Please try this.

[E4gle]

looks like it is working ! Thank you so much for the help.

[IlyaSkriblovsky]

Thanks for using txmongo :-) I'm going to close this issue for now. Feel free to reopen it or create new one if something will go wrong.

[psi29a]

Nice work! We should probably have an example that covers this use case. :)