ICGC-TCGA-PanCancer / DEWrapperWorkflow

This is a workflow which aggregates the results of the EMBL and DKFZ workflows as one SeqWare workflow
0 stars 1 forks source link

is it possible to run this workflow using local path for genome.fa.gz? #13

Closed manuelgarvan closed 8 years ago

manuelgarvan commented 8 years ago

Hi,

I am trying to run DEWrapperWorkflow locally. this is the output:

garvan@docker-host:~$ sudo docker run --rm -h master -it -v /var/run/docker.sock:/var/run/docker.sock \

                              -v /datastore:/datastore \
                              -v /home/garvan/workflow.ini:/workflow.ini \
                              -v /home/garvan/.ssh/gnos.pem:/home/garvan/.ssh/gnos.pem \
            pancancer/pcawg-dewrapper-workflow:1.0.7 \
              seqware bundle launch --dir /workflow/Workflow_Bundle_DEWrapperWorkflow_1.0.7_SeqWare_1.1.1 --engine whitestar --no-metadata --ini /workflow.ini

Performing launch of workflow 'DEWrapperWorkflow' version '1.0.7' [--plugin, net.sourceforge.seqware.pipeline.plugins.BundleManager, --, --install-dir-only, --bundle, /workflow/Workflow_Bundle_DEWrapperWorkflow_1.0.7_SeqWare_1.1.1, --out, /tmp/bundle_m anager3859287578986340414out] Installing Bundle (Working Directory Only) Bundle: /workflow/Workflow_Bundle_DEWrapperWorkflow_1.0.7_SeqWare_1.1.1 Added 'DEWrapperWorkflow' (SWID: 1) Bundle Has Been Installed to the MetaDB and Provisioned to /workflow/Workflow_Bundle_DEWrapperWorkflow_1.0.7_SeqWare_1.1.1! [--plugin, io.seqware.pipeline.plugins.WorkflowScheduler, --, --workflow-accession, 1, --host, master, --out, /tmp/scheduler7584673410809197826out, --i, /workflow.ini, --workflow-engine, whitestar, --no-meta-db, --] Created workflow run with SWID: 62 [--plugin, io.seqware.pipeline.plugins.WorkflowLauncher, --, --launch-scheduled, 62] [2016/07/14 01:00:54] | Number of submitted workflows: 1 Working Run: 62 Valid run by host check: 62 Launching via new launcher: 62 [SeqWare Pipeline] ERROR [2016/07/14 01:00:54] | You need to specify the output prefix for your workflow using either an override parameter at schedule-time or in your workflow INI file as output_prefix [SeqWare Pipeline] ERROR [2016/07/14 01:00:54] | You need to specify the output dir for your workflow using either an override parameter at schedule-time or in your workflow INI file as output_dir! WARNING Running in direct file mode, direct access BAM files will be used and assumed to be full paths

WARNING Running in local file upload mode, analyzed results files will be written to a local directory, you will need to upload to GNOS yourself

Using working directory: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449 [2016/07/14 01:00:54] | Row #0 , Attempt #1 out of 6 : s62_start_0 [2016/07/14 01:00:54] | Submitting 3000M batch with: s62_start_0 [2016/07/14 01:00:54] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_start_0-runner.sh [2016/07/14 01:00:55] | Workflow step succeeded: s62_start_0 [2016/07/14 01:00:55] | Row #1 , Attempt #1 out of 6 : s62_create_dirs_1 [2016/07/14 01:00:55] | Submitting 8192M batch with: s62_create_dirs_1 [2016/07/14 01:00:55] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_create_dirs_1-runner.sh [2016/07/14 01:00:55] | Workflow step succeeded: s62_create_dirs_1 [2016/07/14 01:00:55] | Row #2 , Attempt #1 out of 6 : s62_getEMBLDataFiles_2 [2016/07/14 01:00:55] | Submitting 8192M batch with: s62_getEMBLDataFiles_2 [2016/07/14 01:00:55] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getEMBLDataFiles_2-runner.sh [2016/07/14 01:00:55] | Workflow step succeeded: s62_getEMBLDataFiles_2 [2016/07/14 01:00:55] | Row #3 , Attempt #1 out of 6 : s62_getDKFZDataFiles_3 [2016/07/14 01:00:55] | Submitting 8192M batch with: s62_getDKFZDataFiles_3 [2016/07/14 01:00:55] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3-runner.sh [2016/07/14 01:03:26] | Workflow step failed: s62_getDKFZDataFiles_3 [2016/07/14 01:03:26] | Row #3 , Attempt #2 out of 6 : s62_getDKFZDataFiles_3 [2016/07/14 01:03:26] | Submitting 8192M batch with: s62_getDKFZDataFiles_3 [2016/07/14 01:03:26] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3-runner.sh [2016/07/14 01:05:57] | Workflow step failed: s62_getDKFZDataFiles_3 [2016/07/14 01:05:57] | Row #3 , Attempt #3 out of 6 : s62_getDKFZDataFiles_3 [2016/07/14 01:05:57] | Submitting 8192M batch with: s62_getDKFZDataFiles_3 [2016/07/14 01:05:57] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3-runner.sh [2016/07/14 01:08:28] | Workflow step failed: s62_getDKFZDataFiles_3 [2016/07/14 01:08:28] | Row #3 , Attempt #4 out of 6 : s62_getDKFZDataFiles_3 [2016/07/14 01:08:28] | Submitting 8192M batch with: s62_getDKFZDataFiles_3 [2016/07/14 01:08:28] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3-runner.sh [2016/07/14 01:10:58] | Workflow step failed: s62_getDKFZDataFiles_3 [2016/07/14 01:10:58] | Row #3 , Attempt #5 out of 6 : s62_getDKFZDataFiles_3 [2016/07/14 01:10:58] | Submitting 8192M batch with: s62_getDKFZDataFiles_3 [2016/07/14 01:10:58] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3-runner.sh [2016/07/14 01:13:29] | Workflow step failed: s62_getDKFZDataFiles_3 [2016/07/14 01:13:29] | Row #3 , Attempt #6 out of 6 : s62_getDKFZDataFiles_3 [2016/07/14 01:13:29] | Submitting 8192M batch with: s62_getDKFZDataFiles_3 [2016/07/14 01:13:29] | Running command: bash /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3-runner.sh [2016/07/14 01:16:00] | Workflow step failed: s62_getDKFZDataFiles_3 [2016/07/14 01:16:00] | Setting workflow-run status to failed for: 62 [--plugin, io.seqware.pipeline.plugins.WorkflowWatcher, --, --workflow-run-accession, 62] [SeqWare Pipeline] ERROR [2016/07/14 01:16:00] | You need to specify the output prefix for your workflow using either an override parameter at schedule-time or in your workflow INI file as output_prefix [SeqWare Pipeline] ERROR [2016/07/14 01:16:00] | You need to specify the output dir for your workflow using either an override parameter at schedule-time or in your workflow INI file as output_dir! WARNING Running in direct file mode, direct access BAM files will be used and assumed to be full paths

WARNING Running in local file upload mode, analyzed results files will be written to a local directory, you will need to upload to GNOS yourself

Workflow run 62 is currently failed [--plugin, net.sourceforge.seqware.pipeline.plugins.WorkflowStatusChecker, --, --workflow-run-accession, 62] [2016/07/14 01:16:00] | Output for stdout due to workflow run failure:


Job Name: s62_start_0 Job ID: 458054854 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_start_0.o458054854 Updated: Thu Jul 14 01:00:55 UTC 2016 Contents Excerpt:

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_start_0.stdout


Job Name: s62_create_dirs_1 Job ID: 458055158 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_create_dirs_1.o458055158 Updated: Thu Jul 14 01:00:55 UTC 2016 Contents Excerpt:

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_create_dirs_1.stdout


Job Name: s62_getEMBLDataFiles_2 Job ID: 458055457 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getEMBLDataFiles_2.o458055457 Updated: Thu Jul 14 01:00:55 UTC 2016 Contents Excerpt:

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getEMBLDataFiles_2.stdout


Job Name: s62_getDKFZDataFiles_3 Job ID: 458055740 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.o458055740 Updated: Thu Jul 14 01:03:26 UTC 2016 Contents Excerpt:

The method 'do_run' exited abnormally so the Runner will terminate here! Return value was: 1

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stdout


Job Name: s62_getDKFZDataFiles_3 Job ID: 458206503 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.o458206503 Updated: Thu Jul 14 01:05:57 UTC 2016 Contents Excerpt:

The method 'do_run' exited abnormally so the Runner will terminate here! Return value was: 1

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stdout


Job Name: s62_getDKFZDataFiles_3 Job ID: 458357251 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.o458357251 Updated: Thu Jul 14 01:08:28 UTC 2016 Contents Excerpt:

The method 'do_run' exited abnormally so the Runner will terminate here! Return value was: 1

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stdout


Job Name: s62_getDKFZDataFiles_3 Job ID: 458508012 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.o458508012 Updated: Thu Jul 14 01:10:58 UTC 2016 Contents Excerpt:

The method 'do_run' exited abnormally so the Runner will terminate here! Return value was: 1

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stdout


Job Name: s62_getDKFZDataFiles_3 Job ID: 458658769 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.o458658769 Updated: Thu Jul 14 01:13:29 UTC 2016 Contents Excerpt:

The method 'do_run' exited abnormally so the Runner will terminate here! Return value was: 1

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stdout


Job Name: s62_getDKFZDataFiles_3 Job ID: 458809547 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.o458809547 Updated: Thu Jul 14 01:16:00 UTC 2016 Contents Excerpt:

The method 'do_run' exited abnormally so the Runner will terminate here! Return value was: 1

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stdout

[2016/07/14 01:16:00] | Output for stderr due to workflow run failure:


Job Name: s62_start_0 Job ID: 458054854 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_start_0.e458054854 Updated: Thu Jul 14 01:00:55 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_start_0.sh Command exit code: 0

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_start_0.stderr


Job Name: s62_create_dirs_1 Job ID: 458055158 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_create_dirs_1.e458055158 Updated: Thu Jul 14 01:00:55 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_create_dirs_1.sh Command exit code: 0

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_create_dirs_1.stderr


Job Name: s62_getEMBLDataFiles_2 Job ID: 458055457 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getEMBLDataFiles_2.e458055457 Updated: Thu Jul 14 01:00:55 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getEMBLDataFiles_2.sh Command exit code: 0 /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getEMBLDataFiles_2.stderr


Job Name: s62_getDKFZDataFiles_3 Job ID: 458055740 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.e458055740 Updated: Thu Jul 14 01:03:26 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.sh FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-00-56-726.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-01-46-588.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-02-36-916.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS [SeqWare Pipeline] ERROR [2016/07/14 01:03:26] | The exit value was 1 Command exit code: 1 FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-00-56-726.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-01-46-588.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-02-36-916.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stderr


Job Name: s62_getDKFZDataFiles_3 Job ID: 458206503 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.e458206503 Updated: Thu Jul 14 01:05:57 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.sh FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-03-27-562.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-04-17-549.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-05-07-630.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS [SeqWare Pipeline] ERROR [2016/07/14 01:05:57] | The exit value was 1 Command exit code: 1 FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-03-27-562.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-04-17-549.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-05-07-630.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stderr


Job Name: s62_getDKFZDataFiles_3 Job ID: 458357251 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.e458357251 Updated: Thu Jul 14 01:08:28 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.sh FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-05-57-400.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-06-47-841.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-07-37-288.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS [SeqWare Pipeline] ERROR [2016/07/14 01:08:28] | The exit value was 1 Command exit code: 1 FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-05-57-400.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-06-47-841.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-07-37-288.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stderr


Job Name: s62_getDKFZDataFiles_3 Job ID: 458508012 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.e458508012 Updated: Thu Jul 14 01:10:58 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.sh FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-08-28-387.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-09-18-241.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-10-08-573.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS [SeqWare Pipeline] ERROR [2016/07/14 01:10:58] | The exit value was 1 Command exit code: 1 FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-08-28-387.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-09-18-241.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-10-08-573.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stderr


Job Name: s62_getDKFZDataFiles_3 Job ID: 458658769 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.e458658769 Updated: Thu Jul 14 01:13:29 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.sh FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-10-59-453.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-11-49-846.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-12-39-363.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS [SeqWare Pipeline] ERROR [2016/07/14 01:13:29] | The exit value was 1 Command exit code: 1 FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-10-59-453.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-11-49-846.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-12-39-363.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stderr


Job Name: s62_getDKFZDataFiles_3 Job ID: 458809547 File: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.e458809547 Updated: Thu Jul 14 01:16:00 UTC 2016 Contents Excerpt: Command run: bash -lc /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.sh FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-13-30-599.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-14-20-215.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-15-10-882.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS [SeqWare Pipeline] ERROR [2016/07/14 01:16:00] | The exit value was 1 Command exit code: 1 FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz TIMEOUT: 20 minutes ( 1200000 milliseconds ) STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-13-30-599.log ATTEMPT 1 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-14-20-215.log ATTEMPT 2 OUT OF 3 STARTING DOWNLOAD WITH LOG FILE gtdownload-2016-07-14-01-15-10-882.log ATTEMPT 3 OUT OF 3 FAILED TO DOWNLOAD FILE: 32749c9f-d8aa-4ff5-b32c-296976aec706/dkfz-workflow-dependencies_150318_0951.tar.gz AFTER 3 ATTEMPTS

Full output: /datastore/oozie-b3b394b9-a818-4a0f-afd9-c8dafc41c449/generated-scripts/s62_getDKFZDataFiles_3.stderr

The method 'do_run' exited abnormally so the Runner will terminate here! Return value was: 15 garvan@docker-host:~$

I can run the following commands from my host with no problem: wget https://s3.amazonaws.com/pan-cancer-data/pan-cancer-reference/hs37d5_1000GP.gc wget http://s3.amazonaws.com/pan-cancer-data/pan-cancer-reference/genome.fa.gz

Any idea what is wrong?

thank you very much

manuelgarvan commented 8 years ago

I got this fixed after changing my gnos.pem and the location of the dkfx bundle to the one provided by Annai systems