search

broadinstitute / gatk

Official code repository for GATK versions 4 and up
https://software.broadinstitute.org/gatk
Other
1.72k stars 594 forks source link

gatk 4.1.4.0: haplotypeCaller crashing #6292

Open yuk12 opened 5 years ago

yuk12 commented 5 years ago

Haplotype caller crashing with NULL pointer exception. gatk version 4.1.4.0 Steps followed:

  1. bwa alignment
  2. Sorting
  3. Mark duplicate
  4. BaseRecalibrator
  5. ApplyBQSR
  6. HaplotypeCaller

Here is the haplotype caller log.

Using GATK jar /home/administrator/IGIB/gatk-4.1.4.0/gatk-package-4.1.4.0-local.jar Running: java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/administrator/IGIB/gatk-4.1.4.0/gatk-package-4.1.4.0-local.jar HaplotypeCaller -R /home/administrator/IGIB/samples/Homo_sapiens_assembly38.fasta -I data_gatk/aligned.sort.dup.rg.recal.bam -O data_gatk/aligned.sort.dup.rg.recal.bam.vcf 02:07:46.602 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/administrator/IGIB/gatk-4.1.4.0/gatk-package-4.1.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so Nov 28, 2019 2:07:51 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine INFO: Failed to detect whether we are running on Google Compute Engine. 02:07:51.772 INFO HaplotypeCaller - ------------------------------------------------------------ 02:07:51.773 INFO HaplotypeCaller - The Genome Analysis Toolkit (GATK) v4.1.4.0 02:07:51.773 INFO HaplotypeCaller - For support and documentation go to https://software.broadinstitute.org/gatk/ 02:07:51.773 INFO HaplotypeCaller - Executing as mvasimud@uarch-compression on Linux v4.4.0-142-generic amd64 02:07:51.773 INFO HaplotypeCaller - Java runtime: OpenJDK 64-Bit Server VM v9-internal+0-2016-04-14-195246.buildd.src 02:07:51.773 INFO HaplotypeCaller - Start Date/Time: 28 November 2019 at 2:07:46 AM IST 02:07:51.773 INFO HaplotypeCaller - ------------------------------------------------------------ 02:07:51.773 INFO HaplotypeCaller - ------------------------------------------------------------ 02:07:51.774 INFO HaplotypeCaller - HTSJDK Version: 2.20.3 02:07:51.774 INFO HaplotypeCaller - Picard Version: 2.21.1 02:07:51.774 INFO HaplotypeCaller - HTSJDK Defaults.COMPRESSION_LEVEL : 2 02:07:51.774 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false 02:07:51.774 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true 02:07:51.774 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false 02:07:51.774 INFO HaplotypeCaller - Deflater: IntelDeflater 02:07:51.774 INFO HaplotypeCaller - Inflater: IntelInflater 02:07:51.774 INFO HaplotypeCaller - GCS max retries/reopens: 20 02:07:51.775 INFO HaplotypeCaller - Requester pays: disabled 02:07:51.775 INFO HaplotypeCaller - Initializing engine 02:07:52.246 INFO HaplotypeCaller - Done initializing engine 02:07:52.303 INFO HaplotypeCallerEngine - Disabling physical phasing, which is supported only for reference-model confidence output 02:07:52.312 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/home/administrator/IGIB/gatk-4.1.4.0/gatk-package-4.1.4.0-local.jar!/com/intel/gkl/native/libgkl_utils.so 02:07:52.314 INFO NativeLibraryLoader - Loading libgkl_pairhmm_omp.so from jar:file:/home/administrator/IGIB/gatk-4.1.4.0/gatk-package-4.1.4.0-local.jar!/com/intel/gkl/native/libgkl_pairhmm_omp.so 02:07:52.355 INFO IntelPairHmm - Using CPU-supported AVX-512 instructions 02:07:52.355 INFO IntelPairHmm - Flush-to-zero (FTZ) is enabled when running PairHMM 02:07:52.356 INFO IntelPairHmm - Available threads: 104 02:07:52.356 INFO IntelPairHmm - Requested threads: 4 02:07:52.356 INFO PairHMM - Using the OpenMP multi-threaded AVX-accelerated native PairHMM implementation 02:07:52.408 INFO ProgressMeter - Starting traversal 02:07:52.408 INFO ProgressMeter - Current Locus Elapsed Minutes Regions Processed Regions/Minute 02:07:53.316 WARN InbreedingCoeff - InbreedingCoeff will not be calculated; at least 10 samples must have called genotypes 02:07:53.598 INFO VectorLoglessPairHMM - Time spent in setup for JNI call : 1.49244E-4 02:07:53.598 INFO PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 0.007888748000000001 02:07:53.598 INFO SmithWatermanAligner - Total compute time in java Smith-Waterman : 0.00 sec 02:07:53.598 INFO HaplotypeCaller - Shutting down engine [28 November 2019 at 2:07:53 AM IST] org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller done. Elapsed time: 0.12 minutes. Runtime.totalMemory()=8220835840 java.lang.NullPointerException at org.broadinstitute.hellbender.utils.read.AlignmentUtils.needsConsolidation(AlignmentUtils.java:758) at org.broadinstitute.hellbender.utils.read.AlignmentUtils.consolidateCigar(AlignmentUtils.java:719) at org.broadinstitute.hellbender.utils.read.AlignmentUtils.applyCigarToCigar(AlignmentUtils.java:1290) at org.broadinstitute.hellbender.utils.read.AlignmentUtils.createReadAlignedToRef(AlignmentUtils.java:100) at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.AssemblyBasedCallerUtils.realignReadsToTheirBestHaplotype(AssemblyBasedCallerUtils.java:85) at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:606) at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller.apply(HaplotypeCaller.java:240) at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.processReadShard(AssemblyRegionWalker.java:308) at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.traverse(AssemblyRegionWalker.java:281) at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1048) at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139) at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191) at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210) at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163) at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206) at org.broadinstitute.hellbender.Main.main(Main.java:292)

jackycsie commented 5 years ago

Could you show your command ?

yuk12 commented 4 years ago

Hi, Thanks for the response. Following is the list of commands we used after getting sam file from bwa-mem

gatk=gatk-4.1.4.0/gatk human=/home/administrator/IGIB/samples/Homo_sapiens_assembly38.fasta prefix=aligned

  1. java -jar /home/administrator/IGIB/picard.jar SortSam I=data_gatk/${prefix}.sam O=data_gatk/${prefix}.sort.sam SORT_ORDER=coordinate
  2. java -jar /home/administrator/IGIB/picard.jar MarkDuplicates I=data_gatk/${prefix}.sort.sam O=data_gatk/${prefix}.sort.dup.bam M=data_gatk/marked_dup_metrics.txt
  3. java -jar /home/administrator/IGIB/picard.jar AddOrReplaceReadGroups I=data_gatk/${prefix}.sort.dup.bam O=data_gatk/${prefix}.sort.dup.rg.bam RGID=4 RGLB=lib1 RGPL=illumina RGPU=unit1 RGSM=20
  4. samtools index /home/mvasimud/data_gatk/${prefix}.sort.dup.rg.bam
  5. $gatk BaseRecalibrator -I data_gatk/${prefix}.sort.dup.rg.bam -R $human --known-sites "/home/administrator/IGIB/samples/resources_broad_hg38_v0_Homo_sapiens_assembly38.dbsnp138.vcf" -O data_gatk/basecal.table
  6. $gatk ApplyBQSR -R $human -I data_gatk/${prefix}.sort.dup.rg.bam --bqsr-recal-file data_gatk/basecal.table -O data_gatk/${prefix}.sort.dup.rg.recal.bam
  7. $gatk HaplotypeCaller -R $human -I data_gatk/${prefix}.sort.dup.rg.recal.bam -O data_gatk/${prefix}.sort.dup.rg.recal.bam.vcf
jackycsie commented 4 years ago

I had related experience myself. Then, when I start from "bwa alignment" this step. The problem will be fix. I recommend you can follow command: bwa mem -M -R "@RG\tID:NA12878\tSM:NA12878\tPL:Illumina" -t 80 human_g1k_v37_decoy.fasta NA12878_1.fastq.gz NA12878_2.fastq.gz

Sometime we don't have add this santense"@ RG \tID:NA12878\tSM:NA12878\tPL:Illumina" final running will be stop.

Maybe you can try this method. Thanks

KevinCLydon commented 4 years ago

@yuk12 Would you be able to provide an input to HaplotypeCaller that results in this exception?

Zepeng-Mu commented 4 years ago

Hi, I am getting similar errors but I can get output file. I wonder what this error message actually means? Thanks

KevinCLydon commented 4 years ago

@Zepeng-Mu Could create a forum post (here) explaining what you tried and the specific error you saw?

Zepeng-Mu commented 4 years ago

Hi thanks for your quick response. I created a post and it can be found here: https://gatk.broadinstitute.org/hc/en-us/community/posts/360072760032-HaplotypeCaller-NullPointerException-Error. I saw some similar posts but it seems that there is no solution. Thanks again.