search

niaid / image_portal_workflows

Workflows related to project previously referred to as "Hedwig"
BSD 3-Clause "New" or "Revised" License
5 stars 1 forks source link

rm slurm logs #78

Open philipmac opened 2 years ago

philipmac commented 2 years ago

Once logging is working with single file - rm slurm logs

philipmac commented 11 months ago

on HPC there exists a file that could config this. ~/.config/dask/jobqueue.yaml

See: There exists a param: log-directory: null that's currently commented out. (Thus defaulting the logs to sit on ~.) If this was set to something like

log-directory: ~/slurm_logs

a cron job could run daily and rm everything older than 7 days etc.

philipmac commented 9 months ago

Potentially move these logs could be stored in S3 prior to deletion