Auto scaling in the pipeline without having to create a leader and rsync files

[wshands]

Enables autoscaling on the current VM, so that the leader does not have to be created. User must input several extra parameters to enable autoscaling. The users AWS VM 'Name tag' and security group name must match. The docker option '--net=host ' must be used. AWS credentials must be input. The cluster name must be input; this must match the VM 'Name' tag which must match the security group name.

Example command to run with auto scaling: docker run --net=host -v /var/run/docker.sock:/var/run/docker.sock quay.io/ucsc_cgl/rnaseq-cgl-pipeline:3.4.0a1-1.12.3 --output-basenames 'autoscale_test' 'autoscale_test2' --sample-paired 's3://toil-rnaseq-cloud-staging-area/TEST_R1.fastq.gz,s3://toil-rnaseq-cloud-staging-area/TEST_R2.fastq.gz' 's3://toil-rnaseq-cloud-staging-area/TEST_2_R1.fastq.gz,s3://toil-rnaseq-cloud-staging-area/TEST_2_R2.fastq.gz' --kallisto s3://toil-rnaseq-cloud-staging-area/kallisto_hg38.idx --logDebug --star "" --rsem "" --hera "" --work_mount /home/ubuntu/toil-rnaseq-work/myworkdir --no-clean --auto-scale --output-location s3://toil-rnaseq-cloud-staging-area --credentials-id --credentials-secret-key --cluster-name walt-dev-security-group

[jvivian]

@wshands — I'm in a meeting this whole morning, but I'll try and review this today.

[jvivian]

@wshands — Is this ready to merge?

[wshands]

Yes please merge