Cassandra connection times out when using uwsgi

[laxmanprabhu]

Currently using django-cassandra-engine 0.6.4.4.

When i run my webapp using only django the connection to the cassandra database is ok. But the minute i start the webapp using uwsgi + django, the queries to the database are blocked and finally times out. i tried setting retry_connect = true in the settings file but still the connection set up does not take place.

I see that the previous release has improved connection handling, not sure if the issue i am facing is because of a bug or i am using it in a wrong way.

One last query is under uwsgi when i fork many processes, is the connection to the database supposed to be created on every request, or everytime uwsgi forks the process ?

[r4fek]

Hi @laxmanprabhu

Catch example uwsgi.ini config that works great with latest django-cassandra-engine:

[uwsgi]
# Setuid to the specified user/uid
uid = uwsgi
# Setgid to the specified group/gid
gid = uwsgi
vhost = 
# Load uWSGI plugins (comma-separated)
plugins = python
pidfile = /tmp/app_uwsgi
socket = /run/uwsgi/app/socket
chmod-socket = 666
master = true
enable-threads = true
lazy-apps = true
processes = 40
cpu-affinity = 1
wsgi-file = /var/www/app/django.wsgi
# Every request that will take longer than the seconds specified in the
# harakiri timeout will be dropped and the corresponding worker is thereafter
# recycled.
harakiri = 40
# Log only errors
disable-logging = true
listen = 4096
threads = 2

Could you confirm that similar config does not work for you?

[laxmanprabhu]

I will test this and see and reply back. Thanks

[laxmanprabhu]

I need to dig deeper to find the root cause for this. The connection connects, and i clearly see that the netstat shows that the connections are established. But any queries on execute statements on the cassandra session blocks and waits forever causing timeout. Verified this on 0.6.2 and yields the same result :(

[r4fek]

This doesn't seem like django-cassandra-engine issue. Above uwsgi config works for me since early versions of the wrapper and cassandra-driver. Please take a look at example settings, maybe this would help.

[laxmanprabhu]

Thanks, i will close this issue! At first i thought that django-cassandra-engine was not setting up the cassandra connections. But through netstat its clear that it does set it up. Minute i switch off uwsgi everything works fine. Must be some configuration related thing that i am missing. I will look at the settings file. Thanks again for the help

[laxmanprabhu]

Finally figured it out. Was a uwsgi issue. Somehow the virtualenv in which i was running django had broken uwsgi dependency. uwsgi spawned 5 worker threads but there was just one instance of database connection and as a result the connection just kept timing out.

Thanks again for the help. The configurations helped me nail down the issue. Your help is very much appreciated.

[r4fek]

I'm really glad to hear that. Good luck with your application!

[rinatio]

Hi @r4fek,

Thanks for uwsgi.ini example. We were fighting with similar timeout issue and the solution was to set lazy-apps to true. We could not find reference to this option anywhere else in the django-cassandra-engine documentation. Seems like with lazy-apps uwsgi works even without retry_connect driver connection option you reference in the docs. We found that you recently removed postfork function suggested in the python driver FAQ. Does it mean that django-cassandra-engine will support uwsgi only via lazy-apps? We are asking because uwsgi uses preforking by default and they say both approaches have their pros and cons.

[r4fek]

Hi @rinatio,

You can still use @postfork approach if you like without lazy-apps enabled. However we think enabling it by default is better for various of reasons like f.ex. possibility of using chain reload. Probably I should describe these 2 approaches in Docs so you don't need to struggle with working configs.

[rinatio]

Thank you, @r4fek! We would appreciate if you could describe the difference and give a @postfork usage example. Anyway we'll probably stick with lazy-apps for now.