search

numbas / numbas-lti-provider

An LTI tool provider to run Numbas exams
https://docs.numbas.org.uk/lti/en/latest/
Other
11 stars 12 forks source link

Massive suspend data clogs up the diff_suspend_data task #294

Open christianp opened 8 months ago

christianp commented 8 months ago

We had an attempt whose suspend data was over 20MB long because they wrote python code that printed like 5 million lines, and the output is saved in the pre-submit cache.

When a huey worker tried to diff the suspend data elements, it took so long that the database lock timed out. The attempt was still not diffed, so the next time the diff job came up a worker tried again. This ended up locking up all the workers.

We could try compressing very large suspend data instead of diffing it. Somewhere around 500KB might be a cut-off for compressing instead of diffing. I don't think it's feasible to diff two compressed values.

christianp commented 8 months ago

We should also consider making the programming extension truncate output after a certain point - maybe 1MB.