Currently Arroyo stores all metadata for checkpoints in checkpoints table, overtime it grows upto a significant size which may impact performance.
Immediate solve (by @mwylde ) is to run this query manually ->
DELETE FROM checkpoints
WHERE checkpoints.id != (
SELECT id FROM checkpoints
WHERE job_id = '{{ JOB_ID }}'
ORDER BY finish_time DESC
LIMIT 1
) AND job_id='{{ JOB_ID }}';
Currently Arroyo stores all metadata for checkpoints in
checkpoints
table, overtime it grows upto a significant size which may impact performance.Immediate solve (by @mwylde ) is to run this query manually ->