search

gatk-workflows / five-dollar-genome-analysis-pipeline

Workflows used for WGS data processing -- replaced by https://github.com/gatk-workflows/gatk4-genome-processing-pipeline
https://gatk.broadinstitute.org/hc/en-us
BSD 3-Clause "New" or "Revised" License
57 stars 45 forks source link

SamtoFastq errror when running locally #9

Closed jaideepjoshi closed 5 years ago

jaideepjoshi commented 6 years ago

I keep getting this when running the pipeline locally: Error in writing fastq file /dev/stdout

This is SamtoFastq where FASTQ=/dev/stdout. I have also tried FASTQ=/proc/self/fd/1

Can someone PLEASE tell me what the fix is ?

jaideepjoshi commented 6 years ago

Here are the details: Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell-executions/germline_single_sample_workflow/0c7fec27-ffba-4132-bf87-a3ff2823586a/call-SamToFastqAndBwaMemAndMba/ shard-0/tmp.70736ade Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell-executions/germline_single_sample_workflow/0c7fec27-ffba-4132-bf87-a3ff2823586a/call-SamToFastqAndBwaMemAndMba/ shard-0/tmp.70736ade 17:44:02.776 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/usr/gitc/picard.jar!/com/intel/gkl/native/libgkl_compression.so 17:44:02.785 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/usr/gitc/picard.jar!/com/intel/gkl/native/libgkl_compression.so [Fri Apr 27 17:44:02 UTC 2018] SamToFastq INPUT=/cromwell-executions/germline_single_sample_workflow/0c7fec27-ffba-4132-bf87-a3ff2823586a/call-SamToFastqAndBwaMemA ndMba/shard-0/inputs/mapr/jjoshi.mapr.com/gatk4/five-dollar-genome-analysis-pipeline/NA12878-small/NA12878-small-input-data/HJYFJ.4.NA12878.downsampled.query.sorte d.unmapped.bam FASTQ=/proc/self/fd/1 INTERLEAVE=true INCLUDE_NON_PF_READS=true OUTPUT_PER_RG=false COMPRESS_OUTPUTS_PER_RG=false RG_TAG=PU RE_REVERSE=true CLIPP ING_MIN_LENGTH=0 READ1_TRIM=0 READ2_TRIM=0 INCLUDE_NON_PRIMARY_ALIGNMENTS=false VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_REC ORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false [Fri Apr 27 17:44:02 UTC 2018] Executing as root@9a5e098f97bc on Linux 3.10.0-693.5.2.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_111-8u111-b14-2~bpo8+1-b14; Deflater: Intel; Inflater: Intel; Picard version: 2.15.0-SNAPSHOT [Fri Apr 27 17:44:02 UTC 2018] MergeBamAlignment ADD_PG_TAG_TO_READS=false UNMAPPED_BAM=/cromwell-executions/germline_single_sample_workflow/0c7fec27-ffba-4132-bf8 7-a3ff2823586a/call-SamToFastqAndBwaMemAndMba/shard-0/inputs/mapr/jjoshi.mapr.com/gatk4/five-dollar-genome-analysis-pipeline/NA12878-small/NA12878-small-input-data /HJYFJ.4.NA12878.downsampled.query.sorted.unmapped.bam ALIGNED_BAM=[/dev/stdin] OUTPUT=HJYFJ.4.NA12878.downsampled.query.sorted..aligned.unsorted.bam PROGRAM_RECOR D_ID=bwamem PROGRAM_GROUP_VERSION=0.7.15-r1140 PROGRAM_GROUP_COMMAND_LINE=bwa mem -K 100000000 -p -v 3 -t 16 -Y /cromwell-executions/germline_single_sample_workflo w/0c7fec27-ffba-4132-bf87-a3ff2823586a/call-SamToFastqAndBwaMemAndMba/shard-0/inputs/mapr/jjoshi.mapr.com/gatk4/five-dollar-genome-analysis-pipeline/NA12878-small/ NA12878-small-reference-data/Homo_sapiens_assembly38.fasta PROGRAM_GROUP_NAME=bwamem PAIRED_RUN=true CLIP_ADAPTERS=false IS_BISULFITE_SEQUENCE=false ALIGNEDREADS ONLY=false MAX_INSERTIONS_OR_DELETIONS=-1 ATTRIBUTES_TO_RETAIN=[X0] ATTRIBUTES_TO_REMOVE=[NM, MD] EXPECTED_ORIENTATIONS=[FR] ALIGNER_PROPER_PAIR_FLAGS=true SORT_OR DER=unsorted PRIMARY_ALIGNMENT_STRATEGY=MostDistant ADD_MATE_CIGAR=true UNMAP_CONTAMINANT_READS=true UNMAPPED_READ_STRATEGY=COPY_TO_TAG VALIDATION_STRINGENCY=SILEN T MAX_RECORDS_IN_RAM=2000000 REFERENCE_SEQUENCE=/cromwell-executions/germline_single_sample_workflow/0c7fec27-ffba-4132-bf87-a3ff2823586a/call-SamToFastqAndBwaMemA ndMba/shard-0/inputs/mapr/jjoshi.mapr.com/gatk4/five-dollar-genome-analysis-pipeline/NA12878-small/NA12878-small-reference-data/Homo_sapiens_assembly38.fasta AT TRIBUTES_TO_REVERSE=[OQ, U2] ATTRIBUTES_TO_REVERSE_COMPLEMENT=[E2, SQ] READ1_TRIM=0 READ2_TRIM=0 CLIP_OVERLAPPING_READS=true INCLUDE_SECONDARYALIGNMENTS=true MIN UNCLIPPED_BASES=32 MATCHING_DICTIONARY_TAGS=[M5, LN] VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=2 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=c lient_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false [Fri Apr 27 17:44:02 UTC 2018] Executing as root@9a5e098f97bc on Linux 3.10.0-693.5.2.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_111-8u111-b14-2~bpo8+1-b14; Deflater: Intel; Inflater: Intel; Picard version: 2.15.0-SNAPSHOT INFO 2018-04-27 17:44:03 SamAlignmentMerger Processing SAM file(s): [/dev/stdin] [M::bwa_idx_load_from_disk] read 3171 ALT contigs [W::main_mem] when '-p' is in use, the second query file is ignored. [M::process] read 662252 sequences (100000052 bp)... [M::process] 0 single-end sequences; 662252 paired-end sequences [Fri Apr 27 17:44:35 UTC 2018] picard.sam.SamToFastq done. Elapsed time: 0.55 minutes. Runtime.totalMemory()=5024776192 To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp Exception in thread "main" htsjdk.samtools.SAMException: Error in writing fastq file /proc/self/fd/1 at htsjdk.samtools.fastq.BasicFastqWriter.write(BasicFastqWriter.java:66) at picard.sam.SamToFastq.writeRecord(SamToFastq.java:356) at picard.sam.SamToFastq.doWork(SamToFastq.java:211) at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:268) at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:98) at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:108) [Fri Apr 27 17:44:35 UTC 2018] picard.sam.MergeBamAlignment done. Elapsed time: 0.55 minutes. Runtime.totalMemory()=3014656000 To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp

bshifaw commented 6 years ago

Hi,

Have you tried running the provided json file with sample test file, references, resources? All of which are publicly available for download. Have you tried running the wdl on the cloud or did you start off locally? What command are you using to the execute the wdl? What json file you are using to run the wdl?