Right now @jashapiro runs nextflow locally as his user and it submits jobs which run within Batch. However, if Josh's temporary AWS credentials expire, nextflow is unable to continue submitting jobs.
To prevent this from happening, we should create a separate job queue and compute environment which runs on On Demand instances to run nextflow. This can then use a specific bot user which won't be affected by MFA.
The new flow would then be submit a job to the nextflow-runner queue, which then sends all the nextflow sub-jobs to the currently existing job queue.
Right now @jashapiro runs nextflow locally as his user and it submits jobs which run within Batch. However, if Josh's temporary AWS credentials expire, nextflow is unable to continue submitting jobs.
To prevent this from happening, we should create a separate job queue and compute environment which runs on On Demand instances to run nextflow. This can then use a specific bot user which won't be affected by MFA.
The new flow would then be submit a job to the nextflow-runner queue, which then sends all the nextflow sub-jobs to the currently existing job queue.