nipype / pydra

Pydra Dataflow Engine
https://nipype.github.io/pydra/
Other
119 stars 57 forks source link

Typing of workflow input/output specs #661

Open tclose opened 1 year ago

tclose commented 1 year ago

What would you like changed/added and why?

I would like the option/requirement to declare and type the inputs and outputs of workflows e.g.


wf = pydra.Workflow(name="myworkflow", input_spec=[("in_file", File), ("in_dir", Directory)], output_spec=[("out_file", File)])

Since workflows and tasks can be used interchangeably in Pydra it seems an oversight not to be able to type workflow inputs/outputs. Also, it would enable us to type the LazyField objects from Workflow.lzin and perform type checking on the inputs of workflows, e.g.


    wf = Workflow(name="wf", input_spec=["cmd", "file"])
    wf.inputs.cmd = cmd
    wf.inputs.file = filename

    docky = DockerTask(
        name="docky",
        image="busybox",
        executable=wf.lzin.cmd,  # this can't be type-checked currently
        file=wf.lzin.file,  # nor this
        input_spec=my_input_spec,
        strip=True,
    )
    wf.add(docky)

What would be the benefit? Does the change make something easier to use?

It would help round of dynamic type-checking at construction time, thereby catching more errors before costly workflow runs