search

openedx / event-bus-kafka

Kafka implementation for Open edX event bus
GNU Affero General Public License v3.0
5 stars 5 forks source link

[producer] Message ID not included in audit log when event delivered #128

Closed timmc-edx closed 1 year ago

timmc-edx commented 1 year ago

As of 3.9.2, the consumer's audit logging includes a message_id=<UUID> but the producer is logging message_id=None.

We can still uniquely identify a message within a topic, since we have topic + partition + offset. However, once we start using dead letter queues, retry queues, the outbox pattern—anything where a message might be copied between topics, or be delayed in delivery to a topic—then we'll need to start logging the message_id correctly as well.

robrap commented 1 year ago

Does it include partition and offset, and this is just redundant, but potentially useful data? Thanks.

timmc-edx commented 1 year ago

Yes -- I'll update ticket to explain that.

timmc-edx commented 1 year ago

Turns out evt.headers() is always None in the callback: https://github.com/confluentinc/confluent-kafka-python/issues/574