search

tl-its-umich-edu / my-learning-analytics

My Learning Analytics (MyLA)
Apache License 2.0
36 stars 39 forks source link

Implement scalable event logging #969

Open jennlove-um opened 4 years ago

jennlove-um commented 4 years ago

Describe your problem or feature you'd like added

Currently when requests come in for loading a view we fetch data from DB, event log into a DB table and send the response to UI. We need to rethink how this needs to be designed to fit the scale.

One approach: serve the page/data up to the user then separately log the event

Use a message queue and send events to that. This would be dependent on whether we use AWS or Google.

jennlove-um commented 4 years ago

Related (future) issue: #310 instrument MyLA with caliper

jonespm commented 3 years ago

On the pinax-eventlog page they mention.

For small sites, it should be good enough to use inline but you might want to consider wrapping calls to the log() method and queue them in a job manager like celery or pyres so that the calls become asynchronous.

pushyamig commented 3 years ago

Good Idea, probably for the next release. I wanted the asynchronous logging for better performance. We might want to upgrade the version of Pinax to latest as that now supports Django 3 and MySQL

jennlove-um commented 2 years ago

Additional load testing should be done to understand at what point this becomes an issue for scale.