Delete/Purge sketches and timelines

[berggren]

Batch job to purge sketches and accompanying timelines.

• Archive before deletion
• Export and archive zip file before deletion
• Delete after X days, both sketch, timelines and ES indices

[kiddinn]

This means to essentially:

• Add a new script that checks and executes jobs.
• Add a daily cron to the docker images that runs the new script.
• Add a DB schema for these jobs, so that they can be configured.
• Add an entry into the config file, "PURGE_LABEL = ''" and "DAYS_BEFORE_PURGING" or something like that, which can be used by the script to know what purge actions should take place.
• The script will have sections in it, which defines the actions it can take, each action there (action class) can read from the database and understand what it needs to do (db schema defines "action name", which maps to an action class and then a JSON structure for configuring the job.
• As soon as the script is run, it will check the DB for any actions to be taken. Each action then reads the appropriate configuration and does what it needs to do.
• Pruning is one of these actions, it will start by listing all sketches with the PRUNE label on it, and then check the dates, to see if the label was applied DAYS_BEFORE_PURGING days ago and if so, will export the sketch (that directory needs to be mapped to the host in the docker files) and then purge it.
• Leave room for other administrative actions, such as archiving sketches, etc.

[Kamforka]

Any update on this? I just realized that deleting sketches won't delete underlying timeline indices, thus occupy a lot of storage in a hidden way.