bmtcril
commented
1 year ago There are a few ways to go with this task:
1- As it stands, the branch above has an incomplete start on making multiple backends available using the current edx-platform implementation of coursegraph.
Pros: Quick and relatively easy to get ClickHouse functionality in for future versions of Open edX, no need for another repo, announcing changes, or making folks add another dependency to get the functionality that already exists in the platform.
Cons: It never really made sense for this to live in the platform and architecturally should just be a plugin. It causes neo4j and its dependencies to always be installed even when most deployments never use this feature. Means OARS can't be deployed on older versions of Open edX, when there's really no other reason that should be the case. Bug fixes mean upgrading edx-platform.
2- Break all of coursegraph out of edx-platform into a standalone plugin that can be installed optionally (or is installed by default if we want to keep the current behavior). This would mean creating a new repository and supporting infrastructure.
Pros: Clean separation of coursegraph from the platform, allowing them to be upgraded separately. Older versions of Open edX can install OARS if we maintain all other pieces as tutor plugins. Bug fixes, changes, enhancements can be made separately from the platform.
Cons: Another repository to maintain. It would likely be a breaking change, so communicating that would be important. We would likely need to gather some community feedback since it's an existing production system.
3- Create a separate plugin for just the ClickHouse part of coursegraph and leave the Neo4J implementation alone. Does mean creating and maintaining a new repository, like #2, but spares us the burden of maintaining the existing implementation, trying to make a common interface between the two, and possibly breaking an existing use case. Since the two implementations as they exist in my branch do somewhat different things this is architecturally defensible. I think we would have to rename it away from coursegraph to keep confusion down.
Pros: All of the benefits of #2 but without the additional headaches of moving the existing implementation.
Cons: Two systems that do similar things in different ways, which may be confusing. Still have to maintain another repo.
Once the new Coursegraph-Clickhouse integration lands in edx-platform (hacky, but mostly working branch is here: https://github.com/openedx/edx-platform/tree/bmtcril/coursegraph_csv), we need to remove the code that's currently blowing away and reloading the demo course from the Clickhouse coursegraph tables.