How to set AWS_BATCH_JOB_ATTEMPT value when using curl to submit cromwell jobs to batch?

[apinto-helomics]

I want to set AWS_BATCH_JOB_ATTEMPT in my pipeline. My pipeline is a cromwell wdl pipeline which utilizes AWS batch as the backend. I submit jobs like so

curl -X  POST "http://172.31.77.179:8000/api/workflows/v1"  \
    -H "accept: application/json" \
    -F "workflowSource=@rnaseq_pipeline.wdl" \
    -F "workflowInputs=@rnaseq_pipeline.json" \
    -F "workflowDependencies=@tasks.zip"

From what I read on aws, they seem to set the environment variables for jobs through the websites GUI. I submit my jobs with curl, how would I add the AWS_BATCH_JOB_ATTEMPT value?

Any help would be appreciated, I am not familiar with either cromwell or batch.

[apinto-helomics]

If my question isnt clear enough, let me expand some. I need to set AWS_BATCH_JOB_ATTEMPTS because AWS will terminate my jobs in the middle of them if my spot instance request is outbid (usually the only higher bids are on demand prices). AWS_BATCH_JOB_ATTEMPTS will allow me to tell aws that when I job is stopped for that reason, it will restart the job automatically without me needing to continually monitor it, Is there any way to do this?

[alexiswl]

Hello, the cromwell team use a JIRA now to triage issues - I'm as much of a fan of it as you are. 👎

My suggestion would be to use the the workflow options json to configure your workflow at runtime. See here: https://cromwell.readthedocs.io/en/stable/wf_options/Overview/

This would extend your curl command with -F "workflowOptions=@options.json"

This may be a parameter however that needs to be set in the configuration file that is read by the server when it is launched.

Alexis.