openedx / wg-data

Tracking work and progress of the Open edX Data Working Group
1 stars 2 forks source link

Create an ADR for freezing and strangulating the Open edX native event system #6

Closed e0d closed 1 year ago

e0d commented 2 years ago

Currently, the native event system is required for xAPI events as they are representations of the native events. There are numerous issues with the native event stream, for example, events names don't follow a definite convention, the events are not versioned, some events are invalid JSON.

Rather than fixing all these issues, I propose freezing existing events and declaring the native events as a "use at your own risk zone."

We need to produce and ADR proposing this change and cataloging it's implications.

AC:

bmtcril commented 1 year ago

I think that our way forward here depends a lot on whether we will ever want to support Caliper or other standards. I've been operating on the assumption that Caliper is desirable, and it's largely been implemented beside xAPI, but haven't heard any call from the community for it yet. If we want to support multiple event streams in the future, however, it makes sense to keep the native events and allow for them to be updated / versioned / renamed to convention if changes need to be made to support downstream transformers.

Without Caliper or other event transformers it probably makes sense in the long term to just move to native support for xAPI, but I think the flexibility that we have presently is actually pretty valuable and doesn't force any current operators using native events to change their system.

e0d commented 1 year ago

Yeah, I've probably been insufficiently valuing the value of the native stream for supporting diverse format mappings. It's certainly easier to not remove it. We should probably document this as a decision. If we need a new event, we start with a native event and map from it.

bmtcril commented 1 year ago

I think this is pretty well documented in OEP-26: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0026-arch-realtime-events.html#eventing-components but I'm adding a new task for moving event documentation over to the new site and will make sure that this information is included there.

bmtcril commented 1 year ago

This should be captured well enough in OEP-26 and the existing event documentation to be clear. Currently you cannot create an event in any other format (xAPI / Caliper), only transform existing events to hopefully it's a non-issue for new events.