search

django-nonrel / mongodb-engine

Django MongoDB Backend
http://www.django-nonrel.org/
863 stars 213 forks source link

Connection pooling? #143

Open gpitfield opened 12 years ago

gpitfield commented 12 years ago

Hi all,

I'm noticing that mongod-engine opens up a ton of connections to the database, many of which seem to be left open - at least when using mod_wsgi. I have two questions about this.

First, is there a reason we're not using connection pooling? As I understand it, it's part of PyMongo and is threadsafe (that's about as much as I know); it sure seems like this would make sense to use.

Second, is there a reason we're not doing anything in the DatabaseWrapper close() method, for instance calling something like self.connection.close() ? I'd think that might clean up open connections a bit better, though I haven't tried it yet.

Well, any thoughts or guidance would be appreciated. Thanks!

gpitfield commented 12 years ago

Further to this, what I'm thinking is that we'd open up a persistent Connection in _connect() DatabaseWrapper. Then, in _connect() you'd also call connection.start_request() which will give the caller its own socket. And in _close(), we'd call end_request() which closes that socket/returns it to the pool.

Now, I may very well be missing all sorts of reasons why this is a terrible idea, please do let me know if so :-)

gpitfield commented 12 years ago

Seems like the issues I was seeing were related to this: https://groups.google.com/forum/?fromgroups#!topic/mongodb-user/BOmqWeHw8tc. I still think it might make sense to use pooling, but less of an issue as of now.

jonashaag commented 12 years ago

Sure. I believe there is another issue for this already (but I might also be wrong). I don't think anything hinders us from doing connection pooling.

gpitfield commented 12 years ago

I couldn't find another issue, though I certainly might have missed it...