Workflow as a Service

This is a client and server implementation of the GA4GH Workflow Execution Service 1.0.0 API.

It provides Arvados and Toil backends. It also works with any cwl-runner that supports the CWL standard command line interface: http://www.commonwl.org/v1.0/CommandLineTool.html#Executing_CWL_documents_as_scripts

Installation:

pip install wes-service

Usage

Client configuration

Command line parameter or environment variable.

--host or WES_API_HOST

The host to contact.

--proto or WES_API_PROTO

The protocol (http or https) to use.

--auth or WES_API_AUTH

Credentials. Format is 'Header: value' or just 'value'. If header name is not provided, value goes in the 'Authorization'.

Get service info

$ wes-client --info

Submit a workflow to run:

Attachments must be accessible from the filesystem. Workflow runners may also support http URLs or other storage systems.

$ wes-client --attachments="testdata/dockstore-tool-md5sum.cwl,testdata/md5sum.input" testdata/md5sum.cwl testdata/md5sum.cwl.json

List workflows

$ wes-client --list

Get workflow status

$ wes-client --get <run-id>

Get stderr log from workflow:

$ wes-client --log <run-id>

Server Configuration

Run a standalone server with default cwl-runner backend:

$ wes-server

Run a standalone server with Arvados backend:

$ pip install arvados-cwl-runner
$ wes-server --backend=wes_service.arvados_wes

Run a standalone server with Toil backend:

$ pip install toil[all]
$ wes-server --backend=wes_service.toil_wes --opt extra=--clean=never

Use alternate executable with cwl-runner backend

$ pip install cwltool
$ wes-server --backend=wes_service.cwl_runner --opt runner=cwltool --opt extra=--logLevel=CRITICAL

Pass parameters to cwl-runner

Use "--opt" following by "key=value"

$ wes-server --backend=wes_service.cwl_runner --opt extra=--workDir=/tmp/work

Development

If you would like to develop against workflow-service make sure you pass the provided test and it is flake8 compliant

Install from Source

$ virtualenv venv && source venv/bin/activate && pip install toil[all] && pip install . --process-dependency-links && pip install -r dev-requirements.txt

Running Tests

From path workflow-service run

$ pytest && flake8