feat: auto import course structure on course publish - Githubissues

overhangio / tutor-cairn

Scalable, real-time analytics for Open edX

8 stars 13 forks source link

feat: auto import course structure on course publish #43

Closed Danyal-Faheem closed 2 months ago

Danyal-Faheem commented 3 months ago

Changes

Parse openedx_logs in vector to look for the Updating course overview for <course_id> logs
Send this course_id to a service
Create a new service cairn-watchcourses that listens for incoming events from vector
Trigger importcoursedata script from cairn-watchcourses for the specific course whenever a course is published

Caveats

We utilize batch processing on the vector sinks with a timeout of 300 secs or batch size of 10 events. This means that changes to the course structure may take up to 5 minutes to show up in the superset dashboard.