Closed nsheff closed 1 month ago
@donaldcampbelljr you've been looking at var_templates recently; what do you think of this feature proposal?
Looking into this issue, I had to remind myself what are var_templates
. The documentation exists but is sparse and without concrete examples. Hello Looper has a couple of pipeline interfaces that use them but they are not highlighted anywhere in the documentation.
As a reminder, here are two pipeline_interfaces:
Example using var_templates
:
pipeline_name: example_pipeline
pipeline_type: sample
output_schema: output_schema.yaml
var_templates:
path: "{looper.piface_dir}/pipelines/pipeline1.py"
command_template: >
{pipeline.var_templates.path} --sample-name {sample.sample_name} --req-attr {sample.attr}
Without using var_templates
:
pipeline_name: example_pipeline
pipeline_type: sample
output_schema: output_schema.yaml
command_template: >
python {looper.piface_dir}/count_lines.py {sample.file} {sample.sample_name} {pipestat.results_file}
We would like to enable hierarchy to var_templates such that we could do something like:
var_templates:
my_plugin:
path: "{looper.piface_dir}/pipelines/pipeline1.py"
command_template: >
{pipeline.var_templates.my_plugin.path} --sample-name {sample.sample_name} --req-attr {sample.attr}
I have this working in https://github.com/pepkit/looper/pull/502
I've added recursive functions to expand and render the var_templates so the user can nest them more than 2 levels if they desire.
Right now we have
var_templates
in the pipeline interface, which populates jinja templates. These are useful to provide parameters to plugins, but the names can potentially clash since they're not namespaced by plugin.I propose that var templates should accept a hierarchy, so I can do somethign like:
As is, two plugins can't have the same parameter name, which isn't ideal.