Open philipmac opened 2 years ago
on HPC there exists a file that could config this.
~/.config/dask/jobqueue.yaml
See:
There exists a param:
log-directory: null
that's currently commented out. (Thus defaulting the logs to sit on ~.) If this was set to something like
log-directory: ~/slurm_logs
a cron job could run daily and rm everything older than 7 days etc.
Potentially move these logs could be stored in S3 prior to deletion
Once logging is working with single file - rm slurm logs