Implement recorded run task - Githubissues

whole-tale / gwvolman

Girder Worker Volume Manager Plugin

BSD 3-Clause "New" or "Revised" License

1 stars 2 forks source link

Implement recorded run task #124

Closed craig-willis closed 2 years ago

craig-willis commented 3 years ago

Requirements:

Each run is associated with a version
A new version is created for the run if one doesn't exist
Tale image is built if needed
Run container(s) used to capture provenance, generate reports, etc.
Run executes configured entrypoint and optional tests
Outputs captured in named "run" directory
Active run output can be viewed in the container

The basic flow:

Create run "version"
Build tale image
Add required tools (reprozip, rpz2cpr, geist), ideally as static binaries
Start new container with PTRACE capabilities
Possibly start peer blazegraph instance (so we don't need to install JVM in our container)?
Execute reprozip --trace <entrypoint>
Execute rpz2cpr to generate triples, insert into blazegraph DB
Execute geist to generate reports/visualizations
Run tests
All outputs will be captured in the "run" directory

Outputs include:

Workflow outputs
Complete execution log (stderr/stdout)
Raw reprozip trace output (possibly not stored)
rpz2cpr triples and blazegraph db (local to tale)
Provenance reports/visualizations

craig-willis commented 2 years ago

Implemented in https://github.com/whole-tale/gwvolman/pull/151