search

adamkewley / jobson

A platform for transforming command-line applications into a job service.
Apache License 2.0
256 stars 20 forks source link

Investigate dynamic job outputs #38

Closed adamkewley closed 5 years ago

adamkewley commented 6 years ago

Investigate possibility for dynamic job output lists.

Some projects generate an unknown number of job outputs. The current way they deal with that is by producing a HTML file containing links to the outputs (eugh). Ideally, Jobson should be able to persist an unknown number of outputs.

adamkewley commented 6 years ago

There is another use-case that needs to be handled also: wildcard outputs. Another researcher has jobs that produce files conforming to a pattern (e.g. file*.txt) and Jobson should be intelligent enough to scoop these up.

It is likely that the fix will be to add Regex matching into the path: field, but the fact that a single expectedOutput could produce multiple outputs might thwart the id field, which is currently being used to give Jobson a predictable output ID (for the HTTP API): it might actually be a better idea to drop the id: idea altogether (/w no breaking changes, for pre-1.0), or to add an alternative matchesPattern:-type output, which would be distinct from the default path, id, etc. schema at the moment

adamkewley commented 5 years ago

Kicked to later version, dropped for initial 1.0.0 release.