vatlab / sos

SoS workflow system for daily data analysis
http://vatlab.github.io/sos-docs
BSD 3-Clause "New" or "Revised" License
274 stars 45 forks source link

Alternative task_template? #1531

Open BoPeng opened 8 months ago

BoPeng commented 8 months ago

Currently for each "cluster" we define a task_template with a template for SLURM, PBS etc. The way to specify alternative task_template is through the definition of another "cluster". E.g.

hosts:
    cluster1:
        task_template
    cluster2:
        based_on: hosts.cluster1
        task_template

Or, if the user needs to override some configuraiton, define

hosts:
    cluster1:
        task_template

in its own .sos/hosts.yaml file.

It may be more straightforward to allow multiple templates, such as

hosts:
    cluster1:
        task_template:
        multi_node_task-template:

but an extra parameter would be needed for the task directive, something like

task: queue='cluster1', template='task_template' # default value