Tivix / django-cron

Write cron business logic as a Python class and let this app do the rest! It enables Django projects to schedule cron tasks, tracks their success / failures, manages contention (via a cache) etc. Basically takes care of all the boring work for you :-)
www.tivix.com
MIT License
899 stars 192 forks source link

Logging on Exceptions #117

Open colinhowe opened 7 years ago

colinhowe commented 7 years ago

We use Sentry to capture exceptions. At the moment django_cron captures all unhandled exceptions and logs them to its own error model. Would you accept a PR that also records the unhandled exception using the standard logging framework? This would allow django_cron to integrate with anything that listens to logs.

MaZZly commented 7 years ago

I'd be interested in something similar to get my cronjob failuers to also log into Sentry

cjsoftuk commented 6 years ago

+1 on this. I understand why it captures all exceptions though (as this avoids Cron jobs crashing the django_cron internals and preventing jobs running).

What is probably needed is to find a way to allow Raven (or a.n.other monitoring framework) to hook a job failure.

lauritzen commented 6 years ago

I've accomplished this by adding a post_save hook on the job object which pushes the cron exception message to Sentry:

from django_cron.models import CronJobLog
from django.db.models.signals import post_save
from django.dispatch import receiver
from raven.contrib.django.raven_compat.models import client

@receiver(post_save, sender=CronJobLog, dispatch_uid="propagate_cron_exception")
def propagate_cron_exception(sender, instance, **kwargs):
    if not instance.is_success:
        client.captureMessage("Exception in cron\n\n{}".format(instance.message))
will-emmerson commented 5 years ago

The post_save hook above is clever but unfortunately returns exception as text which doesn't look great in Sentry. I ended up catching the exception, logging it and re-raising it, which definitely is not clever:


from sentry_sdk import capture_exception

class GetPriceImbalance(CronJobBase):
    RUN_EVERY_MINS = 60 * 3

    schedule = Schedule(run_every_mins=RUN_EVERY_MINS)
    code = 'price_imbalance'

    def do(self):
        try:
            get_price_imbalance_data()
        except Exception as e:
            capture_exception(e)
            raise
lauritzen commented 5 years ago

True, which is actually a big shortcoming. You could create a decorator to streamline the code a bit:

from functools import wraps
def sentry_exceptions(func):
    @wraps(func)
    def inner(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except Exception as e:
            capture_exception(e)
            raise

    return inner

Which lets you:


class GetPriceImbalance(CronJobBase):
    RUN_EVERY_MINS = 60 * 3

    schedule = Schedule(run_every_mins=RUN_EVERY_MINS)
    code = 'price_imbalance'

    @sentry_exceptions
    def do(self):
        get_price_imbalance_data()