search

UCSF-Costello-Lab / LG3_Pipeline

The original LG3 pipeline
https://github.com/UCSF-Costello-Lab/LG3_Pipeline
0 stars 0 forks source link

ERROR: MutDet java.lang.RuntimeException: Invalid file pointer: 66987608255590 #28

Open HenrikBengtsson opened 6 years ago

HenrikBengtsson commented 6 years ago

Just recording this one here for future references (in case someone else sees this error too). It looks "random" since the same call worked many times before. I'm rerunning right now.

[cbctest2@cclc01 testrun-4]$ cat MutDet.Patient157t.NOR-Z00599t__REC1-Z00601t.err
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR stack trace 
java.lang.RuntimeException: Invalid file pointer: 66987608255590
        at net.sf.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:477)
        at net.sf.samtools.BAMFileReader$BAMFileIndexIterator.<init>(BAMFileReader.java:577)
        at net.sf.samtools.BAMFileReader.getIterator(BAMFileReader.java:243)
        at net.sf.samtools.SAMFileReader.iterator(SAMFileReader.java:314)
        at org.broadinstitute.sting.gatk.datasources.reads.SAMDataSource.getIterator(SAMDataSource.java:555)
        at org.broadinstitute.sting.gatk.datasources.reads.SAMDataSource.seek(SAMDataSource.java:510)
        at org.broadinstitute.sting.gatk.executive.MicroScheduler.getReadIterator(MicroScheduler.java:211)
        at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:58)
        at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:225)
        at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:104)
        at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:227)
        at org.broadinstitute.sting.gatk.CommandLineGATK.main(CommandLineGATK.java:89)
Caused by: java.io.IOException: Invalid file pointer: 66987608255590
        at net.sf.samtools.util.BlockCompressedInputStream.seek(BlockCompressedInputStream.java:279)
        at net.sf.samtools.BAMFileReader$BAMFileIndexIterator.getNextRecord(BAMFileReader.java:590)
        at net.sf.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:460)
        ... 11 more
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR A GATK RUNTIME ERROR has occurred (version 1.1-37-g5cedb2d):
##### ERROR
##### ERROR Please visit the wiki to see if this is a known problem
##### ERROR If not, please post the error, with stack trace, to the GATK forum
##### ERROR Visit our wiki for extensive documentation http://www.broadinstitute.org/gsa/wiki
##### ERROR Visit our forum to view answers to commonly asked questions http://getsatisfaction.com/gsa
##### ERROR
##### ERROR MESSAGE: Invalid file pointer: 66987608255590
##### ERROR ------------------------------------------------------------------------------------------
[cbctest2@cclc01 testrun-4]$ cat MutDet.Patient157t.NOR-Z00599t__REC1-Z00601t.out
[2018-09-17 23:47:49 PDT] BEGIN: /var/spool/torque/mom_priv/jobs/1218110.cclc01.som.ucsf.edu.SC
Call: /var/spool/torque/mom_priv/jobs/1218110.cclc01.som.ucsf.edu.SC                                                                         
Script: /var/spool/torque/mom_priv/jobs/1218110.cclc01.som.ucsf.edu.SC                                                                       
Arguments:                                                                                                                                   
Settings:                               
- LG3_HOME=/home/shared/cbc/software_cbc/LG3_Pipeline-devel
- LG3_INPUT_ROOT=/cbc/cbctest2/LG3_Pipeline-testing-5c/output
- LG3_OUTPUT_ROOT=/cbc/cbctest2/LG3_Pipeline-testing-5c/output
- EMAIL=henrik.bengtsson+testrun-3@gmail.com
- SCRATCHDIR=/scratch/cbctest2                                                     
- PWD=/home/cbctest2/repositories/testing-readme/testrun-4
- USER=cbctest2
- PBS_NUM_PPN=4
Input:
- PATIENT=Patient157t                                                                                                      
- TUMOR=Z00601t                                                                                                            
- NORMAL=Z00599t                  
- TYPE=REC1            
- CONFIG=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg
- INTERVAL=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/All_exome_targets.extended_200bp.interval_list
- XMX=Xmx8g
- WORKDIR=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
*** Working directory: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
Starting MutDet job on Mon Sep 17 23:47:49 PDT 2018
Patient = Patient157t                                                               
Normal = /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam
Tumor = /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00601t.bwa.realigned.rmDups.recal.bam
Tum. Type = REC1                                                                             
Config = /home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg                 
Interval = /home/jocostello/shared/LG3_Pipeline_HIDE/resources/All_exome_targets.extended_200bp.interval_list
Java Memory = Xmx8g                                                                  
WORKDIR=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect                 
[2018-09-17 23:47:49 PDT] BEGIN: /home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/MutDet.sh
Call: /home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/MutDet.sh                                 
Script: /home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/MutDet.sh                                                                   
Arguments: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00601t.bwa.realigned.rmDups.recal.bam NOR-Z00599t__REC1-Z00601t Patient157t /home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg /home/jocostello/shared/LG3_Pipeline_HIDE/resources/All_exome_targets.extended_200bp.interval_list Xmx8g                   
Settings:                                                    
- LG3_HOME=/home/shared/cbc/software_cbc/LG3_Pipeline-devel   
- LG3_OUTPUT_ROOT=/cbc/cbctest2/LG3_Pipeline-testing-5c/output
- SCRATCHDIR=/scratch/cbctest2                                                     
- PWD=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect                                                       
- USER=cbctest2                                                                                                                          
- PBS_NUM_PPN=4                      
- ncores=4                                                                                               
Input:                                                                                                                     
- nbamfile=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam
- tbamfile=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00601t.bwa.realigned.rmDups.recal.bam                  
- prefix=NOR-Z00599t__REC1-Z00601t                                                                                                           
- patientID=Patient157t                                                                                                                      
- CONFIG=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg                                                 
- ILIST=/home/jocostello/shared/LG3_Pipeline_HIDE/resources/All_exome_targets.extended_200bp.interval_list                                   
- XMX=Xmx8g                                                                                                                                  
Software:                                                                                           
- JAVA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/tools/java/jre1.6.0_27/bin/java
- PYTHON=/usr/bin/python                                                            
- MUTECT=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/tools/muTect-1.0.27783.jar                                     
- FILTER=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/Filter.py                                     
- REORDER=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/vcf_reorder.py
References:                                                                                  
- REF=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/UCSC_HG19_Feb_2009/hg19.fa                  
- DBSNP=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/dbsnp_132.hg19.sorted.vcf                 
- REORDER=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/vcf_reorder.py    
- CONVERT=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/RefSeq.Entrez.txt                
- KINASEDATA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/all_human_kinases.txt      
- COSMICDATA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/CosmicMutantExport_v58_150312.tsv
- CANCERDATA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/SangerCancerGeneCensus_2012-03-15.txt                                
-------------------------------------------------                                                                                            
[MutDet] Mutation detection Mon Sep 17 23:47:49 PDT 2018                                                                                     
-------------------------------------------------                                                                                            
[MutDet] Patient ID: Patient157t                           
[MutDet] Normal Sample: Z00599t                              
[MutDet] Tumor Sample: Z00601t                                
[MutDet] Prefix: NOR-Z00599t__REC1-Z00601t                    
-------------------------------------------------                                  
[MutDet] Normal bam file: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam
[MutDet] Tumor bam file: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00601t.bwa.realigned.rmDups.recal.bam
[MutDet] Java Memory Xmx value: Xmx8g
[MutDet] Working directory: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
-------------------------------------------------                                                                          
[MutDet] Running muTect...                                                                                                 
WARN  02:27:45,707 RestStorageService - Error Response: PUT '/GATK_Run_Reports/CDold49qOkH2mt8mDt8fL1lQxNaPRzXV.report.xml.gz' -- ResponseCode: 403, ResponseStatus: Forbidden, Request Headers: [Content-Length: 2099, Content-MD5: 7RHRMrMtZ2kl6fvdhJY7GQ==, Content-Type: application/octet-stream, x-amz-meta-md5-hash: ed11d132b32d676925e9fbdd84963b19, Date: Tue, 18 Sep 2018 09:27:45 GMT, Authorization: AWS AKIAJXU7VIHBPDW4TDSQ:qV80yYYzMLEE/LKdKablRxPSJlU=, User-Agent: JetS3t/0.8.0 (Linux/2.6.32-504.12.2.el6.664g0000.x86_64; amd64; en; JVM 1.6.0_27), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers: [x-amz-request-id: B278085087C4E6E5, x-amz-id-2: 1i/ImjR0V8rqOhLbDeo2KsalRmuN3IZyagdBCll4LIbb7L4hKLrGcjT+MC/kyTq/G86X55cUGuk=, Content-Type: application/xml, Transfer-Encoding: chunked, Date: Tue, 18 Sep 2018 09:27:43 GMT, Connection: close, Server: AmazonS3]                                                                      
muTect failed                                                                          
End of PBS script!                                                                  
Tue Sep 18 02:27:45 PDT 2018                                                                                             
[2018-09-18 02:27:45 PDT] END: /var/spool/torque/mom_priv/jobs/1218110.cclc01.som.ucsf.edu.SC
PBS Job Id: 1218110.cclc01.som.ucsf.edu
Job Name:   Patient157t.mut
Exec host:  n2/30+n2/31+n2/32+n2/33
Execution terminated
Exit_status=0 resources_used.cput=03:01:53 resources_used.mem=3840672kb resources_used.vmem=11516824kb resources_used.walltime=02:39:57
Error_Path: cclc01.som.ucsf.edu:/home/cbctest2/repositories/testing-readme/testrun-4/MutDet.${PATIENT}.NOR-${NORMAL}__${TYPE}-${TUMOR}.err
Output_Path: cclc01.som.ucsf.edu:/home/cbctest2/repositories/testing-readme/testrun-4/MutDet.${PATIENT}.NOR-${NORMAL}__${TYPE}-${TUMOR}.out
HenrikBengtsson commented 6 years ago

The other sample that ran at the same time (but on n13 compared to above n2; so not same scratch drive) gave another error:

[cbctest2@cclc01 testrun-4]$ cat MutDet.Patient157t.NOR-Z00599t__TUM-Z00600t.err
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR stack trace 
java.lang.RuntimeException: BGZF file has invalid uncompressedLength: -456770510
    at net.sf.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:376)
    at net.sf.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:361)
    at net.sf.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:109)
    at net.sf.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:234)
    at java.io.DataInputStream.read(Unknown Source)
    at net.sf.samtools.util.BinaryCodec.readBytesOrFewer(BinaryCodec.java:394)
    at net.sf.samtools.util.BinaryCodec.readBytes(BinaryCodec.java:371)
    at net.sf.samtools.util.BinaryCodec.readByteBuffer(BinaryCodec.java:480)
    at net.sf.samtools.util.BinaryCodec.readInt(BinaryCodec.java:491)
    at net.sf.samtools.BAMRecordCodec.decode(BAMRecordCodec.java:159)
    at net.sf.samtools.BAMFileReader$BAMFileIterator.getNextRecord(BAMFileReader.java:486)
    at net.sf.samtools.BAMFileReader$BAMFileIndexIterator.getNextRecord(BAMFileReader.java:594)
    at net.sf.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:460)
    at net.sf.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:450)
    at net.sf.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:417)
    at net.sf.samtools.SAMFileReader$AssertableIterator.next(SAMFileReader.java:629)
    at net.sf.samtools.SAMFileReader$AssertableIterator.next(SAMFileReader.java:607)
    at org.broadinstitute.sting.gatk.datasources.reads.SAMDataSource$IntervalOverlapFilteringIterator.advance(SAMDataSource.java:960)
    at org.broadinstitute.sting.gatk.datasources.reads.SAMDataSource$IntervalOverlapFilteringIterator.next(SAMDataSource.java:941)
    at org.broadinstitute.sting.gatk.datasources.reads.SAMDataSource$IntervalOverlapFilteringIterator.next(SAMDataSource.java:876)
    at net.sf.picard.util.PeekableIterator.advance(PeekableIterator.java:71)
    at net.sf.picard.util.PeekableIterator.next(PeekableIterator.java:57)
    at net.sf.picard.sam.MergingSamRecordIterator.next(MergingSamRecordIterator.java:126)
    at net.sf.picard.sam.MergingSamRecordIterator.next(MergingSamRecordIterator.java:39)
    at org.broadinstitute.sting.gatk.iterators.PrivateStringSAMCloseableIterator.next(StingSAMIteratorAdapter.java:100)
    at org.broadinstitute.sting.gatk.iterators.PrivateStringSAMCloseableIterator.next(StingSAMIteratorAdapter.java:84)
    at org.broadinstitute.sting.gatk.datasources.reads.SAMDataSource$ReleasingIterator.next(SAMDataSource.java:861)
    at org.broadinstitute.sting.gatk.datasources.reads.SAMDataSource$ReleasingIterator.next(SAMDataSource.java:827)
    at org.broadinstitute.sting.gatk.iterators.ReadFormattingIterator.next(ReadFormattingIterator.java:81)
    at org.broadinstitute.sting.gatk.iterators.ReadFormattingIterator.next(ReadFormattingIterator.java:14)
    at org.broadinstitute.sting.utils.baq.BAQSamIterator.next(BAQSamIterator.java:50)
    at org.broadinstitute.sting.utils.baq.BAQSamIterator.next(BAQSamIterator.java:15)
    at org.broadinstitute.sting.gatk.filters.CountingFilteringIterator.getNextRecord(CountingFilteringIterator.java:105)
    at org.broadinstitute.sting.gatk.filters.CountingFilteringIterator.next(CountingFilteringIterator.java:81)
    at org.broadinstitute.sting.gatk.filters.CountingFilteringIterator.next(CountingFilteringIterator.java:41)
    at org.broadinstitute.sting.gatk.iterators.PrivateStringSAMCloseableIterator.next(StingSAMIteratorAdapter.java:100)
    at org.broadinstitute.sting.gatk.iterators.PrivateStringSAMCloseableIterator.next(StingSAMIteratorAdapter.java:84)
    at net.sf.picard.util.PeekableIterator.advance(PeekableIterator.java:71)
    at net.sf.picard.util.PeekableIterator.next(PeekableIterator.java:57)
    at org.broadinstitute.sting.gatk.iterators.LocusIteratorByState$ReadStateManager.collectPendingReads(LocusIteratorByState.java:637)
    at org.broadinstitute.sting.gatk.iterators.LocusIteratorByState.lazyLoadNextAlignmentContext(LocusIteratorByState.java:344)
    at org.broadinstitute.sting.gatk.iterators.LocusIteratorByState.hasNext(LocusIteratorByState.java:301)
    at org.broadinstitute.sting.gatk.executive.WindowMaker$WindowMakerIterator.advance(WindowMaker.java:159)
    at org.broadinstitute.sting.gatk.executive.WindowMaker$WindowMakerIterator.next(WindowMaker.java:131)
    at org.broadinstitute.sting.gatk.datasources.providers.LocusView.advance(LocusView.java:156)
    at org.broadinstitute.sting.gatk.datasources.providers.LocusView.hasNextLocus(LocusView.java:117)
    at org.broadinstitute.sting.gatk.datasources.providers.AllLocusView.advance(AllLocusView.java:101)
    at org.broadinstitute.sting.gatk.datasources.providers.AllLocusView.hasNext(AllLocusView.java:59)
    at org.broadinstitute.sting.gatk.traversals.TraverseLoci.traverse(TraverseLoci.java:49)
    at org.broadinstitute.sting.gatk.traversals.TraverseLoci.traverse(TraverseLoci.java:18)
    at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:61)
    at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:225)
    at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:104)
    at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:227)
    at org.broadinstitute.sting.gatk.CommandLineGATK.main(CommandLineGATK.java:89)
Caused by: java.lang.NegativeArraySizeException
    at net.sf.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:374)
    ... 54 more
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR A GATK RUNTIME ERROR has occurred (version 1.1-37-g5cedb2d):
##### ERROR
##### ERROR Please visit the wiki to see if this is a known problem
##### ERROR If not, please post the error, with stack trace, to the GATK forum
##### ERROR Visit our wiki for extensive documentation http://www.broadinstitute.org/gsa/wiki
##### ERROR Visit our forum to view answers to commonly asked questions http://getsatisfaction.com/gsa
##### ERROR
##### ERROR MESSAGE: BGZF file has invalid uncompressedLength: -456770510
##### ERROR ------------------------------------------------------------------------------------------
[cbctest2@cclc01 testrun-4]$ cat MutDet.Patient157t.NOR-Z00599t__TUM-Z00600t.out
[2018-09-17 23:47:48 PDT] BEGIN: /var/spool/torque/mom_priv/jobs/1218109.cclc01.som.ucsf.edu.SC
Call: /var/spool/torque/mom_priv/jobs/1218109.cclc01.som.ucsf.edu.SC
Script: /var/spool/torque/mom_priv/jobs/1218109.cclc01.som.ucsf.edu.SC
Arguments: 
Settings:
- LG3_HOME=/home/shared/cbc/software_cbc/LG3_Pipeline-devel
- LG3_INPUT_ROOT=/cbc/cbctest2/LG3_Pipeline-testing-5c/output
- LG3_OUTPUT_ROOT=/cbc/cbctest2/LG3_Pipeline-testing-5c/output
- EMAIL=henrik.bengtsson+testrun-3@gmail.com
- SCRATCHDIR=/scratch/cbctest2
- PWD=/home/cbctest2/repositories/testing-readme/testrun-4
- USER=cbctest2
- PBS_NUM_PPN=4
Input:
- PATIENT=Patient157t
- TUMOR=Z00600t
- NORMAL=Z00599t
- TYPE=TUM
- CONFIG=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg
- INTERVAL=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/All_exome_targets.extended_200bp.interval_list
- XMX=Xmx8g
- WORKDIR=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
*** Working directory: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
Starting MutDet job on Mon Sep 17 23:47:48 PDT 2018
Patient = Patient157t
Normal = /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam
Tumor = /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00600t.bwa.realigned.rmDups.recal.bam
Tum. Type = TUM
Config = /home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg
Interval = /home/jocostello/shared/LG3_Pipeline_HIDE/resources/All_exome_targets.extended_200bp.interval_list
Java Memory = Xmx8g
WORKDIR=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
[2018-09-17 23:47:48 PDT] BEGIN: /home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/MutDet.sh
Call: /home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/MutDet.sh
Script: /home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/MutDet.sh
Arguments: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00600t.bwa.realigned.rmDups.recal.bam NOR-Z00599t__TUM-Z00600t Patient157t /home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg /home/jocostello/shared/LG3_Pipeline_HIDE/resources/All_exome_targets.extended_200bp.interval_list Xmx8g
Settings:
- LG3_HOME=/home/shared/cbc/software_cbc/LG3_Pipeline-devel
- LG3_OUTPUT_ROOT=/cbc/cbctest2/LG3_Pipeline-testing-5c/output
- SCRATCHDIR=/scratch/cbctest2
- PWD=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
- USER=cbctest2
- PBS_NUM_PPN=4
- ncores=4
Input:
- nbamfile=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam
- tbamfile=/cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00600t.bwa.realigned.rmDups.recal.bam
- prefix=NOR-Z00599t__TUM-Z00600t
- patientID=Patient157t
- CONFIG=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/mutationConfig.cfg
- ILIST=/home/jocostello/shared/LG3_Pipeline_HIDE/resources/All_exome_targets.extended_200bp.interval_list
- XMX=Xmx8g
Software:
- JAVA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/tools/java/jre1.6.0_27/bin/java
- PYTHON=/usr/bin/python
- MUTECT=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/tools/muTect-1.0.27783.jar
- FILTER=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/FilterMutations/Filter.py
- REORDER=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/vcf_reorder.py
References:
- REF=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/UCSC_HG19_Feb_2009/hg19.fa
- DBSNP=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/dbsnp_132.hg19.sorted.vcf
- REORDER=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/scripts/vcf_reorder.py
- CONVERT=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/RefSeq.Entrez.txt
- KINASEDATA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/all_human_kinases.txt
- COSMICDATA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/CosmicMutantExport_v58_150312.tsv
- CANCERDATA=/home/shared/cbc/software_cbc/LG3_Pipeline-devel/resources/SangerCancerGeneCensus_2012-03-15.txt
-------------------------------------------------
[MutDet] Mutation detection Mon Sep 17 23:47:48 PDT 2018
-------------------------------------------------
[MutDet] Patient ID: Patient157t
[MutDet] Normal Sample: Z00599t
[MutDet] Tumor Sample: Z00600t
[MutDet] Prefix: NOR-Z00599t__TUM-Z00600t
-------------------------------------------------
[MutDet] Normal bam file: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00599t.bwa.realigned.rmDups.recal.bam
[MutDet] Tumor bam file: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/exomes_recal/Patient157t/Z00600t.bwa.realigned.rmDups.recal.bam
[MutDet] Java Memory Xmx value: Xmx8g
[MutDet] Working directory: /cbc/cbctest2/LG3_Pipeline-testing-5c/output/LG3/mutations/Patient157t_mutect
-------------------------------------------------
[MutDet] Running muTect...
WARN  02:27:51,813 RestStorageService - Error Response: PUT '/GATK_Run_Reports/jtJqgRTBaAxnRpLzvbgmXQduNlQcsMwz.report.xml.gz' -- ResponseCode: 403, ResponseStatus: Forbidden, Request Headers: [Content-Length: 2908, Content-MD5: hD3oDnKTOZoQ/D5imx7U3w==, Content-Type: application/octet-stream, x-amz-meta-md5-hash: 843de80e7293399a10fc3e629b1ed4df, Date: Tue, 18 Sep 2018 09:27:51 GMT, Authorization: AWS AKIAJXU7VIHBPDW4TDSQ:0zLFSEEkASo+yNzPvpTwQqdcGhM=, User-Agent: JetS3t/0.8.0 (Linux/2.6.32-504.12.2.el6.664g0000.x86_64; amd64; en; JVM 1.6.0_27), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers: [x-amz-request-id: A040201565F86464, x-amz-id-2: DMwks6r5HAXxYiUDJkuaDvYfxwGhN0TsKpEApPNN9BKTX4wrHzK2k2Vp9UQWAL0gyAg2hrQZ9io=, Content-Type: application/xml, Transfer-Encoding: chunked, Date: Tue, 18 Sep 2018 09:27:50 GMT, Connection: close, Server: AmazonS3] 
muTect failed
End of PBS script!
Tue Sep 18 02:27:51 PDT 2018
[2018-09-18 02:27:51 PDT] END: /var/spool/torque/mom_priv/jobs/1218109.cclc01.som.ucsf.edu.SC
PBS Job Id: 1218109.cclc01.som.ucsf.edu
Job Name:   Patient157t.mut
Exec host:  n13/26+n13/27+n13/28+n13/29
Execution terminated
Exit_status=0 resources_used.cput=08:11:26 resources_used.mem=3995568kb resources_used.vmem=11516824kb resources_used.walltime=02:40:04
Error_Path: cclc01.som.ucsf.edu:/home/cbctest2/repositories/testing-readme/testrun-4/MutDet.${PATIENT}.NOR-${NORMAL}__${TYPE}-${TUMOR}.err
Output_Path: cclc01.som.ucsf.edu:/home/cbctest2/repositories/testing-readme/testrun-4/MutDet.${PATIENT}.NOR-${NORMAL}__${TYPE}-${TUMOR}.out
HenrikBengtsson commented 6 years ago

Ah... MutDet works toward the final destination directly, i.e. it does not make sure of the scratch drive. That is, it could be that something went wrong with the destination folder (used by both jobs).

ivan108 commented 6 years ago

I know, writing directly on the head node was not the best idea, may be we should fix it?...

HenrikBengtsson commented 6 years ago

Technically it's writing to the target file server (not the head node), but, I agree, it might be more efficient to work toward local scratch here as well. I've added it to the "Future release (not Next release)" milestone.