va-big-data-genomics / trellis-mvp-functions

Trellis serverless data management framework for variant calling of VA MVP whole-genome sequencing data.
6 stars 1 forks source link

Investigating using Batch to perform virtual machine jobs instead of Dsub #38

Open pbilling opened 1 year ago

pbilling commented 1 year ago

Looks like Batch is designed for clients to interface with it through Python, whereas I had to sort of pull apart the dsub package to launch jobs from Python, and when I do the stdout gets registered as errors in Cloud Logging, which is not ideal.

Batch docs: https://cloud.google.com/batch/docs/create-run-job#create-basic-script