Open kbergin opened 4 years ago
Of note, we’ve made Optimus testing data and it is being used by Hornet in their testing for the DCP. May be helpful re: upstream dependencies and storing truth data. Nick, Kishori, Rex, Saman would be good contacts for Q’s
Postponed because in the short term we are going to use the existing CI infrastructure in Skylab for the HCA pipelines.
To better allow for iterative testing of the Optimus pipeline, we want a workflow test that only runs optimus (vs running as an integration test) and validates the workflow output. Using the dsde-pipelines test framework provides a way to do this.
Why do it? To facilitate iterative development of Optimus Where to start? In dsde-pipelines, subclass
CromwellWorkflowTester
and add a new environment toTestCromwellEnvironment
When is it done? A running workflow test that can be run usingsbt run
Upstream dependencies: requires inputs, options to run Optimus Downstream dependencies: Verify test output with scientific owner, store as truth data, create an Optimus validation WDL, add to nightly workflow test in jenkins. These can all be split into other tasks as necessary.
To solve the problem of cross-repo dependencies, the Optimus test can be written to assume that the skylab repo is checked out in a known relative location to the dsde-pipelines repo. If this repo isn't present the test will fail with an error.
┆Issue is synchronized with this Jira Dev Task