search

FNNDSC / pman

A process management system written in python
MIT License
23 stars 33 forks source link

How do we support SLURM time limit? #191

Open jennydaman opened 2 years ago

jennydaman commented 2 years ago

Time limit is a very influential aspect of a queued job's priority to a SLURM scheduler, so pman should be able to send time limits to Cromwell.

Simplest solution: let CROMWELL_TIMELIMIT be an environment variable for pman, and send it off in the WDL everytime. Multiple pman instances and multiple pfcon instances can be created to define multiple compute environments with different time limits.

Propose spec solution: add to the ChRIS plugin spec, plugin developers must provide an estimate of time needed, and timelimit should be part of resources_dict

jennydaman commented 2 years ago

For now, when configured with the Cromwell backend, pman supports an environment variable configuration TIMELIMIT_MINUTES. See wiki for details.

Multiple instances of pfcon+pman duo pairs can be deloyed where the TIMELIMIT_MINUTES setting is different for each one. These pairs should be registered to CUBE with corresponding names. That way, users can choose time limits as one of few hard-coded values.