Open nsheff opened 6 years ago
I really like that grouping and separation. I'm fine with it being called looper
and think that makes the most sense for now since that's the primary consumer of a pep
, but I think that we could even call it something like engine
, consumer
, or executor
with the idea that a pep
could also be used by a looper
analogue that handles submission of samples to pipelines as defined by a pep
.
Yep, makes sense those would be under looper
. I think that as you point out, a pep should not necessarily be restricted in what sections it have and could allow for any tool that uses a pep to extend it as long as the minimum is there.
ok thanks, I will change the docs to reflect this.
In discussion with @johanneskoester it came up that the
metadata.pipeline_interfaces
section could be considered looper-specific. There are several other looper-specific sections (which I've already divided out in the docs), but this one is unique because it's part of themetadata
section.What about collecting all of these things into a new
looper
section of the project config?The rationale is, each tool that reads a PEP could specify an optional section and do what it wants there. Then you just need to define that section in your PEP if you're using that tool. If it made sense, there could equivalently be introduced a
snakemake
section that could be encode app-specific project configuration (if any exists).So I guess what I'm proposing is to group the sections for
pipeline_args
,pipeline_config
, andcompute
(maybe?), plus the attributemetadata.pipeline_interfaces
, into a new section calledlooper
. I guess the point of this is to have a home formetadata.pipeline_interfaces
. I guessmetadata.results_subdir
andmetadata.submission_subdir
also belong in this class of attributes, so they would followpipeline_interfaces
.Just a thought. We can also just leave it how it is, which is optional and under
metadata
.What do you think, @afrendeiro, @vreuter ?