Open rainsville opened 2 months ago
Hey @rainsville, thanks for writing in.
Could you add debug=True
to your sentry_sdk.init
and check whether you see an event being sent out for the missed check-ins? Trying to determine whether this is an issue between the SDK and Sentry or if the check-ins are not even being emitted by the SDK.
I'd also recommend you try keep_alive=True
. This seemed to improve the situation for folks who reported similar behavior, so it might also help in your case.
How do you use Sentry?
Self-hosted/on-premise
Version
2.5.1
Steps to Reproduce
We have a Celery Redbeat setup with tasks that are scheduled to run every minute (using the interval scheduler). In the Sentry UI, we can see that Sentry expects the tasks to run at
hh:mm:00
. In practice, the tasks are started on time, but around thehh:mm:57
mark.Occasionally, Sentry will report tasks as missing. We have noticed two different scenarios:
Expected Result
The expectation is to not mark the tasks as missed. To account for potential delays in reporting the checkin to Sentry, it may be useful to allow specifying a
checkin_margin
for Celery tasks.Actual Result
Example 1: task is expected to start at
Jun 18, 12:27:00 PM UTC
, is reported as started atJun 18, 12:28:02 PM UTC
and marked as Missed. In the Celery logs, the task is actually started atJun 18, 12:27:57 PM UTC
.Example 2: task is expected to start at
Jun 18, 12:37:00 PM UTC
, starts atJun 18, 12:37:56 PM UTC
, but is still marked as Missed.