Epic: Decommission Legacy OCW Implementations / Architecture Concessions

[rhysyngsun]

Summary

This issue documents parts of our implementation that were implemented as a bridge between the legacy OCW site and ocw-next.

Please make note of what these things need to be removed or cleaned up and any potential issues that could come up. More detail is better.

Code to Remove

• [ ] Remove ocw_import app
  • This can probably be removed entirely

Architectural Cleanup

• [ ] The triggering of WebsiteContent syncing should be moved into django signals
  • This was a design concession to support bulk import of courses without incurring large numbers of github commits. Currently this is triggered by explicit task invocations in the web API. Those should be removed and the calls moved into a signal.

[mbertrand]

Should we remove the single_website_task decorator (website-specific redis lock) when running the content_sync.tasks.sync_all_content task? Should we go back to individual file commits whenever a WebsiteContent object is created/updated instead of running that task?

[rhysyngsun]

Should we remove the single_website_task decorator (website-specific redis lock) when running the content_sync.tasks.sync_all_content task? Should we go back to individual file commits whenever a WebsiteContent object is created/updated instead of running that task?

I think there's still a benefit to doing it this way - it avoids a whole category of issues/quirks that could arise from attempting multiple concurrent writes to the repos. I don't think there's a huge penalty here either because API requests in general should move along quickly.

[mbertrand]

I think there's still a benefit to doing it this way - it avoids a whole category of issues/quirks that could arise from attempting multiple concurrent writes to the repos.

Sounds good to me, in that case should we remove the api/backend methods for making individual commits? GithubBackend.update_content_in_backend, GithubBackend.create_content_in_backend, etc?

[rhysyngsun]

I think there's still a benefit to doing it this way - it avoids a whole category of issues/quirks that could arise from attempting multiple concurrent writes to the repos.

Sounds good to me, in that case should we remove the api/backend methods for making individual commits? GithubBackend.update_content_in_backend, GithubBackend.create_content_in_backend, etc?

Yeah, I think we can just have methods that commit a set of changes, and if we ever need to sync 1 object in the future that's just a set w/ size=1.