Event bus configuration causes events to be duplicated

bmtcril commented 7 months ago

Currently when both xapi and caliper keys are set in EVENT_TRACKING_BACKENDS and pointed to eventtracking.backends.event_bus.EventBusRoutingBackend each event that passes the whitelist will be sent to the same event bus stream twice.

The consumer does not discriminate about which backend it is handling, passing all events to any configured backend. This causes events to be transformed and sent twice. When batching, LRSs will reject the batch due to duplicate event ids. Off the top of my head we should either:

Have separate streams for each backend (better separation of concerns between xapi / caliper, requires a consumer for each) OR
Have one EVENT_TRACKING_BACKENDS for event-routing-backends and let it decide where to send things on event consumption (most efficient, but loses the ability to filter before sending and configure different filters for xapi and caliper?)

Neither of these are great, I'm open to other suggestions but this is a high priority issue.

bmtcril commented 6 months ago

@Ian2012 this was fixed, right?

Ian2012 commented 6 months ago

Yes, can be closed now.

openedx / event-routing-backends

Event bus configuration causes events to be duplicated #410