tl-its-umich-edu / student_explorer

Tool used by Academic advisors to access student weekly performance
Apache License 2.0
10 stars 13 forks source link

Switch cohort loading process to celery rather than request thread #14

Open jonespm opened 6 years ago

jonespm commented 6 years ago

The cohort loading process can take awhile and we've had to implement fixes to work around this like increasing timeouts TLPSO-465. A future version of this tool should pass this work off to a backend process to do the load rather than having the user wait for it.

jonespm commented 5 years ago

I also noticed when investigating the cohort loading process that the actual process is really inefficient.

None of this is batched, it's all single queries. There's no reason for the extra individual selects in step ii. since this doesn't actually function as an update. This would be more efficient if the selects in i. for mentor and student were from the inserts, the inserts in i. were bulk for those missing, then ii. were inserted in