Right Way to Integrate with Celery

Jarch09 commented 7 years ago

python 3.5.2 django 1.10 celery 4.0.0 raven 5.32

Apologies if this is not the appropriate place to post this -- just not sure where else to go. I've tried looking online and have found a number of conflicting sources, including: https://github.com/getsentry/raven-python/issues/922

I set up raven in django by adding

'raven.contrib.django.raven_compat',

to INSTALLED_APPS and by adding the following to my settings file:

RAVEN_CONFIG = { 'dsn': SENTRY_URL, 'CELERY_LOGLEVEL': logging.INFO, }

If I run a sample celery task that raises a runtime error, this set up is sufficient to log that exception (it shows up in my sentry account).

Recently, I had a issue with redis (broker), which led to a situation such that tasks were failing to dispatch, but this error did not show up via sentry.

Here is an example traceback from the log file:

Traceback (most recent call last): File "python3.5/site-packages/kombu/connection.py", line 414, in _reraise_as_library_errors yield File "python3.5/site-packages/kombu/connection.py", line 494, in _ensured return fun(args, kwargs) File "python3.5/site-packages/kombu/messaging.py", line 200, in _publish mandatory=mandatory, immediate=immediate, File "python3.5/site-packages/kombu/transport/virtual/base.py", line 608, in basic_publish return self._put(routing_key, message, kwargs) File "python3.5/site-packages/kombu/transport/redis.py", line 766, in _put client.lpush(self._q_for_pri(queue, pri), dumps(message)) File "python3.5/site-packages/redis/client.py", line 1227, in lpush return self.execute_command('LPUSH', name, values) File "python3.5/site-packages/redis/client.py", line 573, in execute_command return self.parse_response(connection, command_name, **options) File "python3.5/site-packages/redis/client.py", line 585, in parse_response response = connection.read_response() File "python3.5/site-packages/redis/connection.py", line 582, in read_response raise response redis.exceptions.ResponseError: OOM command not allowed when used memory > 'maxmemory'.

How can I configure sentry to properly report this type of error? What is the correct way of configuring sentry with celery 4.0.0+ and django 1.10+?

Thanks for the help

Duologic commented 7 years ago

Put this in your LOGGING dict:

'loggers': {
             ...
             'celery': {'handlers': ['sentry', 'console'],
                        'level': 'WARNING',
                        'propagate': False},
             ...
}

You might also want this logger in your tasks.py:

from celery.utils.log import get_task_logger
logger = get_task_logger(__name__)

Jarch09 commented 6 years ago

@dcramer Can you please confirm that this documentation (link below) covers my issue above? Happy to close the issue.

celery+sentry

getsentry / raven-python

Right Way to Integrate with Celery #1004