search

ICGC-TCGA-PanCancer / CGP-Somatic-Docker

Dockerised and simplified version of SeqWare-CGP-SomaticCore
GNU Affero General Public License v3.0
14 stars 9 forks source link

test data - 1 #4

Closed denis-yuen closed 8 years ago

denis-yuen commented 8 years ago

Currently attempting to execute the workflow with the test data on the branch https://github.com/ICGC-TCGA-PanCancer/CGP-Somatic-Docker/tree/feature/cwl1

Command is 

dockstore tool launch --entry Dockstore.cwl --local-entry --json test1.json

which becomes 

 docker \
    run \
    -i \
...
    --workdir=/var/spool/cwl \
    --read-only=true \
    --user=1000 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/var/spool/cwl \
    quay.io/pancancer/pcawg-sanger-cgp-workflow:feature_cwl1 \
    python \
    /home/seqware/CGP-Somatic-Docker/scripts/run_seqware_workflow.py \
    --tumor \
    /var/lib/cwl/stg842972df-92da-44cb-bd77-8a36a044e395/HCC1143.bam \
    --normal \
    /var/lib/cwl/stga47857ff-ad86-495b-90c2-17c2f4659dfc/HCC1143_BL.bam \
    --refFrom \
    /var/lib/cwl/stg304a71f6-451b-4892-8e49-03e2275b84fa/GRCh37d5_CGP_refBundle.tar.gz \
    --bbFrom \
    /var/lib/cwl/stg2dd32179-5e0c-4bb4-9476-84f20146fcec/GRCh37d5_battenberg.tar.gz

Looking at https://github.com/ICGC-TCGA-PanCancer/CGP-Somatic-Docker/blob/develop/workflow/config/CgpSomaticCore.ini

corresponds to

test1.json.txt with input files retrieved from https://s3-eu-west-1.amazonaws.com/wtsi-pancancer/testdata/HCC1143_ds.tar as a JSON parameter to the CWL file.

Currently fails with the following messages

[2016/08/31 15:27:38] | Row #3 , Attempt #1 out of 1 : s58_compareBamGenotypes_all_4,s58_unpackRef_NA_9,s58_verifyBamHomC
hk_control_6
[2016/08/31 15:27:38] |         Submitting 12000M batch with: s58_compareBamGenotypes_all_4,s58_unpackRef_NA_9,s58_verify
BamHomChk_control_6
[2016/08/31 15:27:38] |         Running command: bash /datastore/oozie-dd8dee89-e0e7-4973-af71-eb97d588a576/generated-scr
ipts/s58_compareBamGenotypes_all_4-runner.sh
[2016/08/31 15:27:38] |         Running command: bash /datastore/oozie-dd8dee89-e0e7-4973-af71-eb97d588a576/generated-scr
ipts/s58_unpackRef_NA_9-runner.sh
[2016/08/31 15:27:38] |         Running command: bash /datastore/oozie-dd8dee89-e0e7-4973-af71-eb97d588a576/generated-scr
ipts/s58_verifyBamHomChk_control_6-runner.sh
[2016/08/31 15:27:43] |         Workflow step failed: s58_compareBamGenotypes_all_4
[2016/08/31 15:27:52] |         Workflow step succeeded: s58_verifyBamHomChk_control_6
Job Name: s58_compareBamGenotypes_all_4
Job ID:   657258007
File:     /datastore/oozie-dd8dee89-e0e7-4973-af71-eb97d588a576/generated-scripts/s58_compareBamGenotypes_all_4.o65725800
7
Updated:  Wed Aug 31 15:27:43 UTC 2016
Contents Excerpt:
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/general.tsv -b /var/spool/cwl/hg19.chr22.5x.normal.bam
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/gender.tsv -b /var/spool/cwl/hg19.chr22.5x.normal.bam
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/general.tsv -b /var/spool/cwl/hg19.chr22.5x.normal2.bam
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/gender.tsv -b /var/spool/cwl/hg19.chr22.5x.normal2.bam
Illegal division by zero at /opt/wtsi-cgp/lib/perl5/Sanger/CGP/NgsQc/CompareGenotypes.pm line 120.
Command exited with non-zero status 255
0.13user 0.00system 0:05.74elapsed 2%CPU (0avgtext+0avgdata 81952maxresident)k
2432inputs+72outputs (6major+7060minor)pagefaults 0swaps
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/general.tsv -b /var/spool/cwl/hg19.chr22.5x.normal.bam
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/gender.tsv -b /var/spool/cwl/hg19.chr22.5x.normal.bam
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/general.tsv -b /var/spool/cwl/hg19.chr22.5x.normal2.bam
Running: /opt/wtsi-cgp/bin/alleleCounter --min-base-qual 20 --min-map-qual 35 --output-file /dev/stdout --loci-file /opt/
wtsi-cgp/lib/perl5/auto/share/dist/Sanger-CGP-NgsQc/genotype/gender.tsv -b /var/spool/cwl/hg19.chr22.5x.normal2.bam
Illegal division by zero at /opt/wtsi-cgp/lib/perl5/Sanger/CGP/NgsQc/CompareGenotypes.pm line 120.
Command exited with non-zero status 255
0.13user 0.00system 0:05.74elapsed 2%CPU (0avgtext+0avgdata 81952maxresident)k
2432inputs+72outputs (6major+7060minor)pagefaults 0swaps
The method 'do_run' exited abnormally so the Runner will terminate here!
Return value was: 1
adamstruck commented 8 years ago

Are the bai files being mounted in with the BAMs?

denis-yuen commented 8 years ago

Yup. This is accomplished by the "secondaryFiles" keyword in the CWL file.

FYI, attempted to re-run on a larger machine, suspected a resource issue. Ended up running into #5 again, so I think we're down to one issue.

ex:

Job Name: s58_ASCAT_ascat_62
Job ID:   817594229
File:     /datastore/oozie-6ddf193c-472f-42ee-a74a-6cc6ce66e7f2/generated-scripts/s58_ASCAT_ascat
_62.o817594229
Updated:  Fri Sep 02 12:38:13 UTC 2016
Contents Excerpt:
Calls: ascat.plotRawData -> png
In addition: Warning message:
In png(filename = paste(ASCATobj$samples[i], ".tumour.png", sep = ""),  :
  unable to open connection to X11 display ''
Execution halted
"cd /var/spool/cwl/0/ascat/tmpAscat/ascat; /opt/wtsi-cgp/bin/Rscript /opt/wtsi-cgp/lib/perl5/auto
/share/module/Sanger-CGP-Ascat-Implement/ascat/runASCAT.R /opt/wtsi-cgp/lib/perl5/auto/share/modu
le/Sanger-CGP-Ascat-Implement/ascat /var/spool/cwl/reference_files/ascat/SnpPositions.tsv /var/sp
ool/cwl/reference_files/ascat/SnpGcCorrections.tsv fc969f27-bdc3-cd03-e040-11ac0c48577e fc969f27-
bdc3-cd03-e040-11ac0c48577e.count fc969f27-bdc6-cd03-e040-11ac0c48577e fc969f27-bdc6-cd03-e040-11
ac0c48577e.count XX /var/spool/cwl/0/ascat/tmpAscat/ascat/fc969f27-bdc3-cd03-e040-11ac0c48577e.Rd
ata" unexpectedly returned exit value 1 at (eval 294) line 13.
 at /opt/wtsi-cgp/lib/perl5/PCAP/Threaded.pm line 190
Command exited with non-zero status 2
597.47user 7.46system 38:15.68elapsed 26%CPU (0avgtext+0avgdata 6048784maxresident)k
662816inputs+209168outputs (130major+435707minor)pagefaults 0swaps
Calls: ascat.plotRawData -> png
In addition: Warning message:
In png(filename = paste(ASCATobj$samples[i], ".tumour.png", sep = ""),  :
  unable to open connection to X11 display ''
Execution halted
"cd /var/spool/cwl/0/ascat/tmpAscat/ascat; /opt/wtsi-cgp/bin/Rscript /opt/wtsi-cgp/lib/perl5/auto
/share/module/Sanger-CGP-Ascat-Implement/ascat/runASCAT.R /opt/wtsi-cgp/lib/perl5/auto/share/modu
le/Sanger-CGP-Ascat-Implement/ascat /var/spool/cwl/reference_files/ascat/SnpPositions.tsv /var/sp
ool/cwl/reference_files/ascat/SnpGcCorrections.tsv fc969f27-bdc3-cd03-e040-11ac0c48577e fc969f27-
bdc3-cd03-e040-11ac0c48577e.count fc969f27-bdc6-cd03-e040-11ac0c48577e fc969f27-bdc6-cd03-e040-11
ac0c48577e.count XX /var/spool/cwl/0/ascat/tmpAscat/ascat/fc969f27-bdc3-cd03-e040-11ac0c48577e.Rd
ata" unexpectedly returned exit value 1 at (eval 294) line 13.
 at /opt/wtsi-cgp/lib/perl5/PCAP/Threaded.pm line 190
Command exited with non-zero status 2
597.47user 7.46system 38:15.68elapsed 26%CPU (0avgtext+0avgdata 6048784maxresident)k
662816inputs+209168outputs (130major+435707minor)pagefaults 0swaps
The method 'do_run' exited abnormally so the Runner will terminate here!
Return value was: 1

Full output: /datastore/oozie-6ddf193c-472f-42ee-a74a-6cc6ce66e7f2/generated-scripts/s58_ASCAT_as
cat_62.stdout