If a course's Mongo content is deleted or moved but the CourseOverview still exists then Figures will still try to process the data. The bulk progress collection call to CourseGradeFactory() will fail, breaking the pipeline execution.
Some initial thoughts on what the solution could looks like
Figures should catch the error, report the error, and move to the next course
The behavior should probably be the same for standalone and multisite Open edX deployments
We need to decide what default and optional alerting looks like
We may want to enhance Figures metrics to have a meta model for the course with
a JSON field to track pipeline errors (minimal data, basic facts, failure on course C for date D) in addition to reporting the error in logging
A field to say to process the course or skip the course for metrics collection
Perhaps instead of storing pipeline error facts in a course meta model, store in the CourseDailyMetrics model for the failure for the day
If a course's Mongo content is deleted or moved but the CourseOverview still exists then Figures will still try to process the data. The bulk progress collection call to
CourseGradeFactory()will fail, breaking the pipeline execution.Some initial thoughts on what the solution could looks like
CourseDailyMetricsmodel for the failure for the day