Some Galaxy jobs create a large amount of data in their working directories. Currently there is no way of monitoring this, instead it requires active investigation to find jobs that are filling the disk. This usually involves looking for jobs that run applications that have been known to create large outputs.
It would be advantageous if monitoring for processes that are chewing up disk space could be automated or turned into a simple gxadmin query that could be run to identify problem jobs. This would require a script that gets a list of running jobs and determines disk usage of their working directories. This will require the ability to access the disk being used and may involve connecting to a remote pulsar destination.
jlqfab
commented
1 year ago
Some Galaxy jobs create a large amount of data in their working directories. Currently there is no way of monitoring this, instead it requires active investigation to find jobs that are filling the disk. This usually involves looking for jobs that run applications that have been known to create large outputs. It would be advantageous if monitoring for processes that are chewing up disk space could be automated or turned into a simple gxadmin query that could be run to identify problem jobs. This would require a script that gets a list of running jobs and determines disk usage of their working directories. This will require the ability to access the disk being used and may involve connecting to a remote pulsar destination.