new looper plugins for sample yaml production

pepkit / looper

A job submitter for Portable Encapsulated Projects

http://looper.databio.org

BSD 2-Clause "Simplified" License

20 stars 7 forks source link

new looper plugins for sample yaml production #299

Closed nsheff closed 3 years ago

nsheff commented 3 years ago

[x] two looper functions wrap each of the peppy sample to_yaml function versions (one with prj embedded and one without),these functions can serve as plugin functions.
[x] looper will no longer direclty call the 'to yaml' function on peppy; it will call via these plugin functions
[x] looper plugin functions will have to define how they want their output file location specified, just as the submission object one does.
[x] we should add a page documenting all these builtin looper plugin functions and how to parameterize them
[x] Template variables should be documented in the pipeline interface specification
[x] A page documenting pre-submission hooks. A section documenting included presubmission hooks.

nsheff commented 3 years ago

Ok I think this is great. @stolarczyk can you go through the docs one more time to see what you think of my changes and make sure everything matches your changes in the plugin code (particularly with respect to sample yaml filepaths and parameters).

stolarczyk commented 3 years ago

I already gave it a read yesterday and made the necessary changes. It looks like noone changed it since then so it should be fine.

nsheff commented 3 years ago

great thanks! I think we're ready to start with the cascade of releases, then

nsheff commented 3 years ago

actually, just caught... shouldn't there be a 'submission subdir' variable int he looper namespace? instead of:

{looper.output_dir}/submission/{sample.sample_name}.yaml it should be {looper.output_dir}/{looper.submission_subdir}/{sample.sample_name}.yaml, or {looper.submission_dir}/{sample.sample_name}.yaml -- that way the plugin parameter would respect the submission subdir provided by the project.