getsentry / raven-python

Raven is the legacy Python client for Sentry (getsentry.com) — replaced by sentry-python
https://sentry.io
BSD 3-Clause "New" or "Revised" License
1.68k stars 657 forks source link

Since upgrading to raven 5.60 sentry is full of celery info level messages #649

Open pg1671 opened 9 years ago

pg1671 commented 9 years ago

I have my loggers set to not send info level messages to sentry but since upgrading to 5.60 I am now getting lots of info level messages dominating my sentry issue feed.

There is a note in the change log about a new celery related feature in 5.60 so wondering if this is related.

downgrading to 5.50 stops the info messages.

ferenc-radius commented 9 years ago

I got the same issue,

I followed the instruction from this page: http://raven.readthedocs.org/en/latest/integrations/celery.html

I'm downgrading till this got fixed

arthurk commented 9 years ago

We updated to raven 5.6.0 today and saw the same issue. Sentry started to rate-limit/reject a lot of messages due to the amount of INFO logs.

dcramer commented 9 years ago

I haven't had time to look into this yet, but odds are we removed some behavior that prevented allowing things through and this opened up a can of worms.

We've had issues internally getting LOGGING sorted out, especially with Celery workers.

mrkschan commented 9 years ago

I got the same issue, and figured out why.

At https://github.com/getsentry/raven-python/blob/d65cc551ca25327c365d4dbd02aaccba7c0077d6/raven/contrib/celery/__init__.py#L67, the Raven LOGLEVEL is overwritten by the Celery LOGLEVEL. Find below an example.

# Celery app.py
class Celery(celery.Celery):
    def on_configure(self):
        import logging
        client = raven.Client(settings.SENTRY_DSN)

        register_logger_signal(client, loglevel=logging.WARNING)
        register_signal(client)

app = Celery('app')

# Run Celery by
celery worker -a app -l INFO

With this setup, I find that Celery INFO logs are sent to Sentry server which I don't intend to do so.

dcramer commented 9 years ago

We need to sort out this behavior yet, but for now I've reverted the Celery change sin master.

mrkschan commented 9 years ago

FYI, I added a workaround myself like below so that I keep my Celery LOGLEVEL at INFO while sending WARNING to Sentry.

        # TODO: Drop this workaround after bugfix of Raven 5.6.0.
        # Ref - https://github.com/getsentry/raven-python/issues/649
        class WarningFilter(logging.Filter):
            def filter(self, record):
                return record.levelno >= logging.WARNING

        def sentry_logging_filter(sender, logger, loglevel, logfile, format,
                                  colorize, **kw):
            for h in logger.handlers:
                if type(h) == SentryHandler:
                    filter_ = WarningFilter()
                    h.addFilter(filter_)
                    return False
        after_setup_logger.connect(sentry_logging_filter, weak=False)
        # End workaround for bug in Raven 5.6.0