riga / law

Build large-scale task workflows: luigi + job submission + remote targets + environment sandboxing using Docker/Singularity
http://law.readthedocs.io
BSD 3-Clause "New" or "Revised" License
96 stars 39 forks source link

Passthrough workflow type. #155

Closed riga closed 1 year ago

riga commented 1 year ago

This PR changes the way the --workflow TYPE parameter is passed between (workflow) tasks. Considering the following scenario,

graph TD
    A("TaskA (HTCondorWorkflow, LocalWorkflow)")
    B("TaskB (LocalWorkflow)")
    C("TaskC (HTCondorWorkflow, LocalWorkflow)")

    A --> B
    B --> C

running law run TaskC --workflow htcondor will currently

In the current state, this could be avoided by configuring the requirements from TaskB to TaskA in a way that the task-level CLI parameter --TaskA-workflow would have preference, and then running law run TaskC --workflow htcondor --TaskA-workflow htcondor, but this is somewhat cumbersome and unexpected.

This PR stores the requested workflow type in a different parameter than the effective one and passes the former upstream (instead of the latter one). This is the new default behavior, which can be toggled back to the above description by setting passthrough_requested_workflow = False on task class level.