search

broadinstitute / PANOPLY

Repository for the Broad Institute Proteogenomic Data Analysis Center (PGDAC) established by the NIH Clinical Proteomics Tumor Analysis Consortium (CPTAC)
Other
33 stars 15 forks source link

Migrate tasks/workflow to Hydrant #3

Closed drmani closed 5 years ago

drmani commented 6 years ago

Currently, PGDAC tasks and workflows are defined in an ad-hoc manner using the gdac-firecloud "make" system. This is sub-optimal, and can results in inconsistencies of naming, etc. Migrate all tasks and workflows to Hydrant, and have a systematic way of modifying and uploading methods to FireCloud.

ramanibk commented 5 years ago

A separate folder hydrant was added to the repository which contains sub-directories for each task in the pgdac_main pipeline.

  1. It consists of set-up scripts which can be run for each individual task or new tasks to create hydrant working directory with templates for wdls, dockerfiles, etc. This script will build and push dockers to docker hub automatically provided it is given correct parameters. The source code has to be present in the src/task_name directory. Additionally, user must be docker-authenticated via terminal.
  2. wdl has to be modified manually still.
  3. Once modified, the tasks can be validated via $ hydrant validate. This step will create inputs.json file in the tests subdirectory, which also needs to be modified manually for the purposes of local testing.
  4. Once updated, the entire docker + wdl bundle can be tested using $ hydrant test.
  5. Lastly, once everything seems error free, the corresponding method can be installed using $ hydrant install.