The script is activated once a day on slurm nodes. It's meant to work with user-specific cache directories and node-global scratch directories. When the amount of free space drops below a threshold, the following steps take place:
A slurm command to drain the local node is issued in order to prevent the node from accepting jobs while the cleanup takes place.
If processing a user directory, it is cleaned up only if the user has no jobs in the current node's queue.
If processing a node-global directory, it is cleaned up only if the current node has no running jobs.
A slurm command to put the node back into service is issued.
User directories are specified by passing their parent directory to the --userdir-root option. The value must be a path containing subdirectories named after users. For example, if --userdir-root=/local/usercache is passed, the directory structure must be as follows:
/local/usercache/
|- alice/
|- bob/
|- ...
Only the contents of the user directories deleted, not the directories themselves.
Node-global directories are straightforward. For example, if --globaldir=/local/scratch is passed, the contents of /local/scratch are deleted. The directory itself is not deleted.
The script is activated once a day on slurm nodes. It's meant to work with user-specific cache directories and node-global scratch directories. When the amount of free space drops below a threshold, the following steps take place:
User directories are specified by passing their parent directory to the
--userdir-rootoption. The value must be a path containing subdirectories named after users. For example, if--userdir-root=/local/usercacheis passed, the directory structure must be as follows:Only the contents of the user directories deleted, not the directories themselves.
Node-global directories are straightforward. For example, if
--globaldir=/local/scratchis passed, the contents of/local/scratchare deleted. The directory itself is not deleted.