felipemontoya
commented
9 hours ago I'm all for the end-goal of this proposal.
I think every provider has met issues with course-discovery and would like to see a replacement that is actually part of the core, is organization agnostic and can be extended. I know as a provider I definitely want that. I even proposed a pattern of hooks to address this issue among others when I was looking at the possible roadmap for hooks for this year and beyond.
Implement Hook Pattern in Core Components of IDAs Implement a standardized pattern of hooks in the platform's core components and other IDAs, which currently lack extensibility entry points such as course about page, course progress, search, homepage, login/registration, learner-dashboard, profile page, certificates, credentials, programs page, ecommerce APIs, discovery, catalog and enterprise views. This will involve adding filters at the beginning of each process and firing events at the end, allowing organizations to customize core logic to fit client needs.
Taken from this hooks roadmap document
Something that this proposal is not considering is how the information of the catalog will be displayed. I understand that 2U uses the catalog as an API to feed into their marketing site which is a whole other system also not in the core. Perhaps we could leverage a Course search MFE such as the one proposed here
Some specific comments:
-
new catalog app: should this proposal take off, I believe we should put the core primitives like programs or catalog and write the most basic implementation possible. Then leave every bit of product development to plugins. I don't think that requirements for courses or anything so specific should be implemented in core. Mostly a models and calls to extension points should live there.
-
extending core models: I love the idea. I however don't know how we could bring this into the hooks framework in a performant way. I'll be happy to collaborate on this until we reach a good solution.
-
roles: this is a whole project on itself.
bradenmacdonald
Proposal Date
23 September 2024
Ticket Acceptance Date
23 October 2024
Technology Removal Date
2025-2027
First Open edX Named Release Without This Functionality
TBD
Rationale
The Course Discovery Service provides a variety of optional functionality in Open edX, some of which is required for other services like the edx-enterprise suite. However, the fundamental architecture of the discovery service has proven to be problematic, and it is currently a limiting factor that makes it much more difficult than it should be to develop and integrate custom functionality for the Open edX platform such as ecommerce, enterprise integrations, subscriptions, and programs.
Here are some of the problems with its current architecture:
Coursemodel in its MySQL database, and theCourseRunDocumentin Elasticsearch). Though there are various systems for keeping this data in sync with Studio (which holds the actual "source of truth"), they are imperfect and synchronization issues are a frequent source of bugs and errors.Organizationvs. CoreOrganization,Modevs.CourseMode,Personvs.UserProfile,CourseEditorvs.CourseAccessRole). Often, these models have either subtle or substantial differences from the Studio/LMS version, and there is no guarantee that they are in sync at all.Kitchen Sink / edX-Specific Functionality: The Course Discovery Service includes, in no particular order:
paid-bootcampandbootcamp-2u(not to be confused with Tracks, Modes, or Seat Types which are all related)SubjectTopicFactCertificateInfoIt may be the case that the platform lacked ready-to-use extension points at the time these features were required, but they should have been built as extensions regardless.
Removal
The entire Course Discovery microservice and the corresponding code in edx-platform that integrates with it. However, see below - this is not going away anytime soon.
Replacement
To rectify this situation, this proposal calls for a series of improvements to the Open edX core (edx-platform and openedx-learning, etc.) that add a "catalog" feature and focus on improving platform extensibility, so that the kitchen sink of features listed above could be largely built using plugins. At the same time, the Discovery service would be officially deprecated.
New Flexible Catalog App
To replace one of the core features of the Discovery Service, a new "Catalog" application should be built into the core platform. The purpose of this app is to provide a unified API for answering the question of "What courses are available to me?". It should be extensible using plugins so that things like Subscriptions and Enterprise Licensing can be accounted for.
This catalog app may also have a flexible concept of "programs" which are essentially groups of related courses, but where each program can define requirements like "pass course A then B, then either C and/or D in order to complete the program." Significant product research and planning is required to define such requirements in more detail.
The catalog app should be developed with "micro-courses" in mind as well - courses significantly smaller than the average MOOC and more like a "Section" of today's typical courses.
The detailed design of the catalog app will happen in future ADRs (and/or an OEP) if this DEPR proposal is accepted. There is already a draft of one initial ADR to add an edx-platform representation of "Catalog Courses".
Extending Core Models
Many of the other "kitchen sink" features listed above were developed in the Discovery service because there wasn't a better place to put them at the time. Instead of finding the least bad place to put specialized functionality like this, we should modify the platform so that the core models can be extended by plugins.
For example, a plugin should be able to add custom fields to the
UserProfilemodel, and have these fields (optionall) show up anywhere user profiles are seen or edited. Likewise the core Course model should be extensible, so a plugin could add (e.g.) a "Price" field to each course and choose to have it show up in the catalog API data, the default catalog UI (when user's browse for courses), and the default course "Advanced Settings" UI. The Open edX Hooks Extension Framework already provides some mechanisms for this, but others will need to be proposed in future ADRs/OEPs.Roles & Permissions
Achieving a good implementation of a core catalog app and extensible core models will almost definitely depend on a major refactor of roles and permissions, the lack of which has been a long-standing pain point in Open edX.
Deprecation
Deprecating the Discovery Service now means three things:
First, proposing any new functionality for the Open edX Project whose implementation depends on the Discovery Service would be rejected.
Second, maintenance of the Discovery Service would become entirely the responsibility of those organizations that [still] use it (e.g. 2U).
Third, eventually the corresponding hooks and integration code in
edx-platform(e.g. sync code, "programs cache, etc.) would be removed. This is likely many years away (see below). However, if this DEPR is accepted, we will make an effort to flag all of this code and corresponding APIs so that it's clear it will be eventually removed, and should not be used for implementing new functionality.Migration
For any functionality that currently depends on the Discovery Service (e.g. "Programs"), there will be no effort to remove corresponding integration code from edx-platform until an alternative approach along the lines proposed has been developed, stablized, and released for enough time that organizations have a reasonable migration path. Details of that will depend on the implementation. Those who currently depend on the discovery service for functionality critical to their organization may continue to use it for foreseeable future, provided they are able to contribute to its ongoing maintenance.
The goal of this DEPR proposal is not to take the discovery service away as soon as possible, but to focus community resources on developing a better alternative, and to encourage stakeholders to avoid building any new features on top of the discovery service from this point forward.