WorkflowConfiguration FK to workflow

[johnbradley]

New Process for creating a job

User finds a workflow version

User lists the workflows via GET /api/v2/workflows/ Example response:

[
    {
        "id": 1,
        "name": "WES GATK4 Preprocessing",
        "versions": [
            1
        ],
        "tag": "wes-gatk4-preprocessing"
    }
]

User lists the workflow_versions via GET /api/v2/workflow-versions/ Example response:

[
    {
        "id": 1,
        "workflow": 1,
        "name": "WES GATK4 Preprocessing",
        "description": "WES GATK4 Preprocessing",
        "object_name": "#main",
        "created": "2018-11-14T19:50:35.431516Z",
        "url": "https://github.com/Duke-GCB/bespin-cwl/releases/download/v0.9.4/exomeseq-gatk4-preprocessing.cwl",
        "version": 1,
        "methods_document": null,
        "fields": [
            ...
            {
                "name": "threads",
                "type": "int"
            }
        ],
        "tag": "wes-gatk4-preprocessing/v1"
    }
]

User finds a workflow configuration

User lists the workflow configurations that match the chose workflow via GET /api/v2/workflow-configurations/?workflow=1 Example response:

[
    {
        "id": 1,
        "tag": "b37-human-xgen",
        "name": "b37-human-xgen",
        "system_job_order": {
            "threads": 32,
            ...
        },
        "workflow": 1,
        "default_vm_strategy": 1,
        "share_group": 1
    }
]

User determines a tag that describes the desired job

User combines a workflow version tag with a workflow configuration tag. User combines workflow version tag for wes-gatk4-preprocessing/v1 and workflow configuration tag for b37-human-xgen. So the resulting tag is wes-gatk4-preprocessing/v1/b37-human-xgen.

User inits a job-template based on a tag that describes the desired job

User sends POST /api/v2/job-templates/init/ with payload {"workflow_tag": "wes-gatk4-preprocessing/v1/b37-human-xgen"} Example response:

{
    "workflow_tag": "wes-gatk4-preprocessing/v1/b37-human-xgen",
    "name": "<String Value>",
    "fund_code": "<String Value>",
    "job_order": {
        "library": "<String Value>",
        ...
    }
}

User finds a share group

User fetches share groups via GET /api/v2/share-groups/ Example response:

[
    {
        "id": 1,
        "name": "ExomeSeqCheckers",
        "email": "checker@exome.seq",
        "users": [
            1
        ]
    }
]

User fills this id for the create job endpoint as share_group. User can optionally lookup a vm strategy via GET to /api/v2/vm-strategies/ as job_vm_strategy

User fills in the job template and creates a job

User sends POST /api/v2/job-templates/create-job/ with edited payload from init-job-file.

Modifications after review

Using tag instead of workflow_tag. Removed share_group option for creating jobs. Workflow configuration only has tag field now.

[johnbradley]

Reconsidering passing version# when creating a new workflow version.

[johnbradley]

I'm going to leave the workflow version number as a passed in field. For production use we want to base this version on part of the git release tag, but for development just incrementing it is fine.

[johnbradley]

Perhaps combined_tag would be a better field name here. (I do think workflow_tag is a good name for this concept)

So would we change workflow_tag to combined_tag in the JobTemplate file as well?