AWS: Error creating reconfigured-script

[henriqueribeiro]

I'm running gatk-sv workflows with AWS backend and I'm facing some issues on scatter tasks. It seems that for some of the tasks, the reconfigured-script is bad constructed. Below is an excerpt from the script:

#!/bin/bash

{
echo '*** LOCALIZING INPUTS ***'
if [ ! -d /tmp/scratch ]; then mkdir /tmp/scratch && chmod 777 /tmp/scratch; fi
cd /tmp/scratch
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-48/SR00c.NA20802.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-48/SR00c.NA20802.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-26/SR00c.NA19350.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-26/SR00c.NA19350.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-37/SR00c.NA19818.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-37/SR00c.NA19818.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-33/SR00c.NA19679.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-33/SR00c.NA19679.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-23/SR00c.NA19102.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-23/SR00c.NA19102.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-10/SR00c.NA18539.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-10/SR00c.NA18539.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-5/SR00c.NA12489.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-5/SR00c.NA12489.txt.gz

/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-MergeSRFilesByContig/shard-10/write_lines_68566dce2f453fb61e4c5ebe7cfd828f.tmp /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-MergeSRFilesByContig/shard-10/write_lines_68566dce2f453fb61e4c5ebe7cfd828f.tmp
sed -i 's#/cromwell_root#/tmp/scratch#g' /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-MergeSRFilesByContig/shard-10/write_lines_68566dce2f453fb61e4c5ebe7cfd828f.tmp

/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-40/SR00c.NA20320.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-40/SR00c.NA20320.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-13/SR00c.NA18560.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-13/SR00c.NA18560.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-42/SR00c.NA20346.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-42/SR00c.NA20346.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-22/SR00c.NA19062.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-22/SR00c.NA19062.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-6/SR00c.NA12872.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-6/SR00c.NA12872.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-8/SR00c.NA18507.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-8/SR00c.NA18507.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-4/SR00c.NA12340.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-4/SR00c.NA12340.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-53/SR00c.NA21122.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-53/SR00c.NA21122.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-15/SR00c.NA18923.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-15/SR00c.NA18923.txt.gz.tbi
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-17/SR00c.NA18945.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-17/SR00c.NA18945.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-50/SR00c.NA20869.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-50/SR00c.NA20869.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-20/SR00c.NA19001.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-20/SR00c.NA19001.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-43/SR00c.NA20509.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-43/SR00c.NA20509.txt.gz
/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz /tmp/scratch/s3:/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz

(sorry for the long log) As you can see, in the last line, the output path of s3 copy is malformed, there is an 's3:/' lost there. This causes the whole pipeline to fail. I already tried several times and sometimes it happens, sometimes don't. Also when it happens, it's not always in the same shard. Do you have any ideia why this is happening? Thanks in advance

[markjschreiber]

Hi,

This looks like a bug in the generation of the script. Any chance you can share the WDL file with me?

On Mon, Nov 30, 2020 at 7:35 AM henriqueribeiro notifications@github.com wrote:

I'm running gatk-sv workflows with AWS backend and I'm facing some issues on scatter tasks. It seems that for some of the tasks, the reconfigured-script is bad constructed. Below is an excerpt from the script:

!/bin/bash

{echo ' LOCALIZING INPUTS 'if [ ! -d /tmp/scratch ]; then mkdir /tmp/scratch && chmod 777 /tmp/scratch; ficd /tmp/scratch /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-48/SR00c.NA20802.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-48/SR00c.NA20802.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-26/SR00c.NA19350.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-26/SR00c.NA19350.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-37/SR00c.NA19818.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-37/SR00c.NA19818.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-33/SR00c.NA19679.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-33/SR00c.NA19679.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-41/SR00c.NA20321.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-23/SR00c.NA19102.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-23/SR00c.NA19102.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-31/SR00c.NA19661.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-10/SR00c.NA18539.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-10/SR00c.NA18539.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-5/SR00c.NA12489.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-5/SR00c.NA12489.txt.gz

/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-MergeSRFilesByContig/shard-10/write_lines_68566dce2f453fb61e4c5ebe7cfd828f.tmp /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-MergeSRFilesByContig/shard-10/write_lines_68566dce2f453fb61e4c5ebe7cfd828f.tmp sed -i 's#/cromwell_root#/tmp/scratch#g' /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-MergeSRFilesByContig/shard-10/write_lines_68566dce2f453fb61e4c5ebe7cfd828f.tmp

/usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-40/SR00c.NA20320.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-40/SR00c.NA20320.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-13/SR00c.NA18560.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-13/SR00c.NA18560.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-42/SR00c.NA20346.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-42/SR00c.NA20346.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-22/SR00c.NA19062.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-22/SR00c.NA19062.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-6/SR00c.NA12872.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-6/SR00c.NA12872.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-8/SR00c.NA18507.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-8/SR00c.NA18507.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-4/SR00c.NA12340.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-4/SR00c.NA12340.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-53/SR00c.NA21122.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-53/SR00c.NA21122.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-15/SR00c.NA18923.txt.gz.tbi /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-15/SR00c.NA18923.txt.gz.tbi /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-17/SR00c.NA18945.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-17/SR00c.NA18945.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-16/SR00c.NA18941.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-50/SR00c.NA20869.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-50/SR00c.NA20869.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-20/SR00c.NA19001.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-20/SR00c.NA19001.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-43/SR00c.NA20509.txt.gz /tmp/scratch/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-43/SR00c.NA20509.txt.gz /usr/local/aws-cli/v2/current/bin/aws s3 cp --no-progress s3://aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz /tmp/scratch/s3:/aaaaaaa-gwf-core/cromwell-execution/Module00c/ab433d98-4b83-4bc5-bb21-ec8f057d81af/call-EvidenceMerging/EvidenceMerging/f4fa818c-5f46-4891-a26a-0f0368fef639/call-SetSampleIdSR/shard-32/SR00c.NA19678.txt.gz

{code}

(sorry for the long log) As you can see, in the last line, the output path of s3 copy is malformed, there is an 's3:/' lost there. This causes the whole pipeline to fail. I already tried several times and sometimes it happens, sometimes don't. Also when it happens, it's not always in the same shard. Do you have any ideia why this is happening? Thanks in advance

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/broadinstitute/cromwell/issues/6106, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF2E6ELH2MI44ZN2D3LYJZTSSOGSVANCNFSM4UHQIFCA .

[henriqueribeiro]

Sure. The main WDL is this one: https://github.com/broadinstitute/gatk-sv/blob/master/wdl/Module00c.wdl

In this specific case, the tasks that are failing are on BatchEvidenceMerging.wdl

[biofilos]

I have a similar problem, though in my case, part of the reconfigured-script.sh says
/tmp/scratch//tmp/scratch/cromwell.results/cromwell-execution/Variants/e83d62c8-c896-45ee-806c-30c75d0e183f/call-coCleaning/coCleaning/dee9d53e-f865-436e-ad1b-499d848852ac/call-applyRecalibration/shard-0/recalibrated.bai
any ideas on how to resolve this?
Thanks!