getsentry / arroyo

A library to build streaming applications that consume from and produce to Kafka.
https://getsentry.github.io/arroyo/
Apache License 2.0
41 stars 7 forks source link

feat: Add optional received_p99 timestamp to commit log #295

Closed lynnagara closed 11 months ago

lynnagara commented 11 months ago

The value from the received field can be used in the future for subscription scheduling if this is provided. This is better than the orig_message_ts field as received is assigned at the very start of the pipeline when Sentry receives the event (as opposed to when Snuba gets the event). Switching to this field means any delays in ingestion will be properly accounted for when determining the window on which to schedule subscriptions.

This PR also: