NoHostAvailable using celery with several workers

[nq-ebaratte]

According to the FAQ, I start celery with:

import os
import sys

from celery import Celery
from django.conf import settings
from celery.signals import worker_process_init
from django.db import connections

connection = connections['cassandra']

@worker_process_init.connect
def connect_db(**kwargs):
    connection.reconnect()

app = Celery()
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(settings.INSTALLED_APPS, related_name='construct_workers')

When using several workers, an running several tasks involving requests to cassandra, only the first started worker will get a functionning connection. Other workers fail with:

NoHostAvailable: ('Unable to complete the operation against any hosts', '{<Host: XXX.XXX.XXX.XXX:datacenter1>: NoConnectionsAvailable()}')

[tushar-thantharate]

Facing same issue with cassandra and celery workers. I have used below links settings:

http://cqlengine.readthedocs.io/en/latest/topics/third_party.html?highlight=celery

[tushar-thantharate]

@r4fek any input on above mention issue?

[r4fek]

Sorry for the delay. Following code works great for me on production with no errors:

db_connection = connections['cassandra']

@worker_process_init.connect
def connect_db(**_):
    db_connection.reconnect()

@worker_shutdown.connect
def disconnect(**_):
    db_connection.connection.close_all()

[tushar-thantharate]

import os from celery import Celery from celery.signals import worker_process_init, beat_init, worker_shutdown from django.conf import settings from django.db import connections

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'tushar.settings') db_connection = connections['cassandra']

"""Initialize worker context for both standard and periodic tasks""" @beat_init @worker_process_init.connect def connect_db(**kwargs): db_connection.reconnect()

@worker_shutdown.connect def disconnect(**kwargs): db_connection.connection.close_all()

app = Celery('iproctor') app.config_from_object('django.conf:settings') app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

@r4fek Using above code as mentioned by you didn't work. Still having same issue

"NoHostAvailable('Unable to complete the operation against any hosts', '{<Host: XX.XX.XX.XX datacenter1>: NoConnectionsAvailable()}')"}

[tushar-thantharate]

@nq-ebaratte were you able to resolve the issue? TIA