feat: ingest external course APIs

[asadali145]

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/4032

Description (What does it do?)

Ingest external course APIs for Emeritus.

This PR ingests the External Course API for Emeritus and creates/updates xPro data like Course, CourseRun, ExternalCoursePage, Topics, WhoShouldEnrollPage, and LearningOutcomesPage.

Here are a few things that are done intentionally:

• Existing course readable IDs and course run courseware IDs are not changed, They are used in Salesforce and other platforms.
• Code might seem complex due to existing bad data and bad API data.
• If a course page exists, we only update in case when some of the fields are empty to avoid overriding the manual changes.
• Course Run Update: We only update the dates for course runs if they do not match.
• I have added 2 tests for fetch_emeritus_course_runs to cater to the success and failure in multi-request cases.

Celery beat schedule is pending. I will add that after confirming with Bon.

How can this be tested?

Note: Before reviewing this PR, go through the following things in the linked issue:

• the linked code snippet to read the Emeritus API, Would be better to run the snippet and get the API response in the form of a JSON file.
• Discussion on the issue
• The analytics dashboard for Emeritus as mentioned in the issue description.

Steps Please read all the steps before testing.

• Checkout master and create some courses, the course runs, external course pages and subpages using the prod data for Emeritus. This will help us identify any issues with existing courses.
  • DO ADD Course.external_course_id and CourseRun.external_course_run_id as we only match these fields with the Emeritus course and course run codes to avoid any confusion. If you do not add these keys then the ingestion will assume them missing and will create new courses and course runs.
  • Try to create as much data as possible
  • You can leave a few runs of a course to check that new runs are created after ingestion.
  • Similarly, we can also leave the creation of External course pages and subpages like Who should enroll and learning outcomes page for some courses.
  • Courses: https://xpro.mit.edu/admin/courses/course/?platform__id__exact=1
  • Course Runs: https://xpro.mit.edu/admin/courses/courserun/?course__platform__id__exact=1
• As we have enough data for Emeritus, now check out this branch.
• Run the management command ./manage.py sync_external_course_runs --vendor_name emeritus
• Monitor the logs of the celery task task_sync_emeritus_course_runs, the task should be successful and it should create or update the required data using the API response.

[arslanashraf7]

@asadali145 Another thing:

• We'll need to create a PR in ol-infra to add the API key otherwise this breaks the web server with improperly configured values.
• We'll also need a PR to enable this feat flag in RC and Prod, But for the sake of testing we can manually add the in Heroku RC when we need.

[asadali145]

@cachob Could you please review these Screenshots:

Course Page created with API data:

Existing Emeritus Course:

New Emeritus Course created with API data:

Existing Course Run:

New Course Run created with API data: