open-oni / oni-agent

Simple service for remote execution of batch-related management commands for ONI
0 stars 0 forks source link

Purge cached jobs regularly #1

Open jechols opened 1 month ago

jechols commented 1 month ago

In theory if a lot of jobs come through, and the agent is running for a long enough time, its RAM usage will become unsustainable. The storage of job logs is "forever" (until the daemon is stopped, anyway), which is completely unnecessary.

If we just add a date to the job data, we can truncate data after a short while (e.g., after a day, store just the basic job data and maybe 50 or so lines of the job's log), and purge them entirely after a longer time, like a week.