karlokr-p commented 1 year ago

Checklist

[x] I have included information about relevant versions
[x] I have verified that the issue persists when using the master branch of Faust.

Steps to reproduce

Initialise sentry normally using the built-in function setup_sentry()

Expected behavior

There should be no warnings about deprecated stuff

The sentry environment is properly set (ie, differing between production/staging environments)

Actual behavior

Many warnings are seen in the logs

The sentry environment is not properly set and all errors are sent to the same environment in sentry, so I'm unable to differentiate between which environment the error occurred on.

Full traceback

TypeError: As of 3.10, the *loop* parameter was removed from Queue() since it is no longer necessary
  File "raven/base.py", line 732, in send_remote
    transport = self.remote.get_transport()
  File "raven/conf/remote.py", line 71, in get_transport
    self._transport = self._transport_cls(**self.options)
  File "raven_aiohttp.py", line 195, in __init__
    self._queue = asyncio.Queue(maxsize=qsize, loop=self._loop)
  File "asyncio/queues.py", line 34, in __init__
    super().__init__(loop=loop)
  File "asyncio/mixins.py", line 17, in __init__
    raise TypeError(

Known cause

The raven and raven-aiohttp packages have not been updated since 2019. These packages do not support python 3.10, hence the constant warnings about the loop parameter. According sentry, raven has been deprecated https://docs.sentry.io/clients/python/

My solution

Initialise the current version of sentry-sdk manually:

sentry_sdk.init(
    dsn=settings.SENTRY_DSN,
    integrations=[_sdk_aiohttp.AioHttpIntegration()],
    traces_sample_rate=settings.SENTRY_TRACES_SAMPLE_RATE,
    environment=settings.SENTRY_ENVIRONMENT,
    release=settings.RELEASE_VERSION,
)

Versions

Python version: 3.10.9
Faust version: All versions since 3.10 support was added
Operating system: Docker (built from debian-slim-bullseye image)
Kafka version: confluentinc/cp-kafka 7.2.2
RocksDB version (if applicable): N/A

wbarnha commented 1 year ago

Unfortunately I'm not familiar with sentry. This was implemented well before my time as a maintainer. Could you please open a PR with your solution or provide a more broad example?

karlokr-p commented 1 year ago

The setup_sentry() function makes use of a custom sentry handler, FaustSentryHandler which includes some additional logic:

# 1) We override SentryHandler to write internal log messages
#    mto sys.__stderr__ instead of sys.stderr, as the latter
#    is redirected to a logger, causing noisy internal logs
#    to be sent to Sentry.
#
# 2) We augment can_record to skip logging for CancelledError
#    exceptions that have a ``.is_expected`` attribute set to True.
#    That way once a CancelledError showing up in Sentry is
#    investigated and resolved, we can silence that particular
#    error in the future.

Which I am not sure how to implement without digging into the sentry-sdk documentation. My solution does not include this custom handler so I'm not sure my solution is good enough to include as a PR, unfortunately. I can attempt to re-write this custom handler, but it may take some time

harochau commented 1 year ago

Can confirm, it is better to remove the built-in (outdated) sentry integration and point users to the stock solution. I spent some time trying to make the built-in solution to work, only to discover it does not work for any Python version newer than 3.9. The stock solution is:

from sentry_sdk.integrations.aiohttp import AioHttpIntegration
from sentry_sdk.integrations.logging import EventHandler

faust_app = faust.App(...)
sentry_sdk.init(
    environment=settings.environment_name,
    dsn=settings.sentry_dsn,
    integrations=[AioHttpIntegration()],
    traces_sample_rate=1.0,
)
sentry_log_handler = EventHandler(level=WARNING)  # send errors and warnings to sentry
faust_app.conf.loghandlers.append(sentry_log_handler)

wbarnha commented 1 year ago

Can confirm, it is better to remove the built-in (outdated) sentry integration and point users to the stock solution. I spent some time trying to make the built-in solution to work, only to discover it does not work for any Python version newer than 3.9. The stock solution is:
from sentry_sdk.integrations.aiohttp import AioHttpIntegration
from sentry_sdk.integrations.logging import EventHandler

faust_app = faust.App(...)
sentry_sdk.init(
    environment=settings.environment_name,
    dsn=settings.sentry_dsn,
    integrations=[AioHttpIntegration()],
    traces_sample_rate=1.0,
)
sentry_log_handler = EventHandler(level=WARNING)  # send errors and warnings to sentry
faust_app.conf.loghandlers.append(sentry_log_handler)

Thanks for the example, I'm probably going to deprecate the existing sentry integrations and replace it with their suggested solution.

tworedz commented 2 months ago

@wbarnha What is the final solution here? In Python 3.11, it is not working because of the raven_aiohttp library, resulting in the following error: AttributeError: module 'asyncio' has no attribute 'coroutine'

I can contribute by either completely removing the Sentry integration or fixing it with the actual sentry-sdk library.

faust-streaming / faust

Sentry initialisation uses outdated raven packages resulting in constant warnings and inability to set environments #487