search

nf-core / sarek

Analysis pipeline to detect germline or somatic variants (pre-processing, variant calling and annotation) from WGS / targeted sequencing
https://nf-co.re/sarek
MIT License
388 stars 401 forks source link

ERROR > SAREK_BAM_VARIANT_CALLING_GERMLINE (MANTA) #1611

Open kchilkert opened 1 month ago

kchilkert commented 1 month ago

Description of the bug

I've run into an issue running the Sarek pipeline with Sea Urchin genome data. I would like to run Manta to find all variants from short read DNA sequencing data. I have already ran this same data with GATK Haplotypecaller successfully, however, I'm running into an issue that I do not fully understand.

I made a custom config file as I had 8hr time limit issues running GATK, creating that custom config file fixed that so I decided to use it again here for Manta.

Any help is appreciated.

Command used and terminal output

Launching Pipeline:
nextflow run nf-core/sarek -r 3.3.2 -name Urchin_manta_data_1 -profile singularity -params-file nf-params.json --skip_tools baserecalibrator --genome null --igenomes_ignore --fasta /path/to/my/file/ragtag.HiFi.polished.simplified.filtered.fasta -c /path/to/my/file/customsarek.config

Output Error:
ERROR ~ Error executing process > 'NFCORE_SAREK:SAREK:BAM_VARIANT_CALLING_GERMLINE_ALL:BAM_VARIANT_CALLING_GERMLINE_MANTA:MANTA_GERMLINE (illumina)'

Caused by:
  Process `NFCORE_SAREK:SAREK:BAM_VARIANT_CALLING_GERMLINE_ALL:BAM_VARIANT_CALLING_GERMLINE_MANTA:MANTA_GERMLINE (illumina)` terminated with an error exit status (1)

Command executed:

  configManta.py         --bam illumina.md.cram         --reference ragtag.HiFi.polished.simplified.filtered.fasta         --runDir manta         --callRegions [ragtag.HiFi.polished.simplified.filtered.fasta].bed.gz

  python manta/runWorkflow.py -m local -j 30

  mv manta/results/variants/candidateSmallIndels.vcf.gz         illumina.manta.candidate_small_indels.vcf.gz
  mv manta/results/variants/candidateSmallIndels.vcf.gz.tbi         illumina.manta.candidate_small_indels.vcf.gz.tbi
  mv manta/results/variants/candidateSV.vcf.gz         illumina.manta.candidate_sv.vcf.gz
  mv manta/results/variants/candidateSV.vcf.gz.tbi         illumina.manta.candidate_sv.vcf.gz.tbi
  mv manta/results/variants/diploidSV.vcf.gz         illumina.manta.diploid_sv.vcf.gz
  mv manta/results/variants/diploidSV.vcf.gz.tbi         illumina.manta.diploid_sv.vcf.gz.tbi

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_SAREK:SAREK:BAM_VARIANT_CALLING_GERMLINE_ALL:BAM_VARIANT_CALLING_GERMLINE_MANTA:MANTA_GERMLINE":
      manta: $( configManta.py --version )
  END_VERSIONS

Command exit status:
  1

Command output:

  Successfully created workflow run script.
  To execute the workflow, run the following script and set appropriate options:

  manta/runWorkflow.py
Command error:
  [2024-08-02T04:59:34.017008Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_069_scf00000069-1-4126316_0000' launched from master workflow
  [2024-08-02T04:59:34.245234Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_065_scf00000065-1-4189768_0000' launched from master workflow
  [2024-08-02T04:59:34.373047Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_253_seq00000024_RagTag-1-1873092_0000' launched from master workflow
  [2024-08-02T04:59:34.500506Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_556_seq00000481_RagTag-1-4291023_0000' launched from master workflow
  [2024-08-02T04:59:34.777749Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_071_scf00000071-1-14326224_0000' launched from master workflow
  [2024-08-02T04:59:34.777817Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_107_scf00000107-1-3365541_0000' launched from master workflow
  [2024-08-02T04:59:35.004529Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_045_scf00000045-1-4371124_0000' launched from master workflow
  [2024-08-02T04:59:35.887639Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_279_seq00000069_RagTag-1-3676942_0000' launched from master workflow
  [2024-08-02T04:59:36.871557Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeLocusGraph_chromId_028_scf00000028-1-7379030_0000' launched from master workflow
  [2024-08-02T04:59:36.881890Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Launching sub-workflow task: 'mergeLocusGraphInputList' from master workflow
  [2024-08-02T04:59:36.882175Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:mergeLocusGraphInputList] Starting task specification for sub-workflow
  [2024-08-02T04:59:36.892291Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:mergeLocusGraphInputList] Finished task specification for sub-workflow
  [2024-08-02T04:59:36.898523Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed sub-workflow task: 'mergeLocusGraphInputList' launched from master workflow
  [2024-08-02T04:59:36.898938Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Launching command task: 'mergeLocusGraph' from master workflow
  [2024-08-02T04:59:36.917906Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:mergeLocusGraph] Task initiated on local node
  [2024-08-02T05:00:08.376843Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'mergeLocusGraph' launched from master workflow
  [2024-08-02T05:00:08.401917Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Launching command task: 'removeTmpDir' from master workflow
  [2024-08-02T05:00:08.402227Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Launching command task: 'checkLocusGraph' from master workflow
  [2024-08-02T05:00:08.402413Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Launching command task: 'locusGraphStats' from master workflow
  [2024-08-02T05:00:08.422485Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:removeTmpDir] Task initiated on local node
  [2024-08-02T05:00:08.424033Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:checkLocusGraph] Task initiated on local node
  [2024-08-02T05:00:08.424707Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:locusGraphStats] Task initiated on local node
  [2024-08-02T05:00:11.628345Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'locusGraphStats' launched from master workflow
  [2024-08-02T05:00:12.746512Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'checkLocusGraph' launched from master workflow
  [2024-08-02T05:00:12.746982Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Launching command task: 'makeHyGenDir' from master workflow
  [2024-08-02T05:00:12.761186Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:makeHyGenDir] Task initiated on local node
  [2024-08-02T05:00:12.862868Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'makeHyGenDir' launched from master workflow
  [2024-08-02T05:00:16.037506Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Completed command task: 'removeTmpDir' launched from master workflow
  [2024-08-02T05:00:16.037941Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] Launching command task: 'generateCandidateSV_0000' from master workflow
  [2024-08-02T05:00:16.053038Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskRunner:generateCandidateSV_0000] Task initiated on local node
  [2024-08-02T05:07:12.656332Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] Failed to complete command task: 'generateCandidateSV_0000' launched from master workflow, error code: 1, command: '/usr/local/share/manta-1.6.0-1/libexec/Genera$
  [2024-08-02T05:07:12.658080Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] Error Message:
  [2024-08-02T05:07:12.658130Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] Anomalous task wrapper stderr output. Wrapper signal file: 'manta/workspace/pyflow.data/logs/tmp/taskWrapperLogs/004/513/pyflowTaskWra$
  [2024-08-02T05:07:12.658174Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] Logging 5 line(s) of task wrapper log output below:
  [2024-08-02T05:07:12.658207Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:00:16.109282Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_0000] $
  [2024-08-02T05:07:12.658249Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:00:16.130044Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_0000] $
  [2024-08-02T05:07:12.658283Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:07:12.580347Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_0000] $
  [2024-08-02T05:07:12.658312Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:07:12.589605Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_0000] $
  [2024-08-02T05:07:12.658342Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] Last 0 stderr lines from task (of 0 total lines):
  [2024-08-02T05:07:12.658377Z] [hpc-compute-p27.cm.cluster] [4159_1] [TaskManager] [ERROR] Shutting down task submission. Waiting for remaining tasks to complete.
  [2024-08-02T05:07:22.358650Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] Workflow terminated due to the following task errors:
  [2024-08-02T05:07:22.358699Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] Failed to complete command task: 'generateCandidateSV_0000' launched from master workflow, error code: 1, command: '/usr/local/share/manta-1.6.0-1/libexec/Gen$
  [2024-08-02T05:07:22.358716Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] Error Message:
  [2024-08-02T05:07:22.358740Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] Anomalous task wrapper stderr output. Wrapper signal file: 'manta/workspace/pyflow.data/logs/tmp/taskWrapperLogs/004/513/pyflowTask$
  [2024-08-02T05:07:22.358749Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] Logging 5 line(s) of task wrapper log output below:
  [2024-08-02T05:07:22.358769Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:00:16.109282Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_000$
  [2024-08-02T05:07:22.358778Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:00:16.130044Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_000$
  [2024-08-02T05:07:22.358789Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:07:12.580347Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_000$
  [2024-08-02T05:07:22.358797Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] [2024-08-02T05:07:12.589605Z] [hpc-compute-p27.cm.cluster] [4159_1] [pyflowTaskWrapper:generateCandidateSV_000$
  [2024-08-02T05:07:22.358806Z] [hpc-compute-p27.cm.cluster] [4159_1] [WorkflowRunner] [ERROR] [generateCandidateSV_0000] [taskWrapper-stderr] Last 0 stderr lines from task (of 0 total lines):

Relevant files

params { max_memory = 128.GB max_cpus = 32 max_time = 144.h }

process { withName:'NFCORE_SAREK:SAREK:BAM_VARIANT_CALLING_GERMLINE_ALL:BAM_VARIANT_CALLING_HAPLOTYPECALLER:GATK4_HAPLOTYPECALLER' { time = 24.h } }

System information

Nextflow: 23.10.1 Hardward: HPC Executor: Slurm Container: Singularity OS: macOS nf-core/sarek: 3.3.2

FriederikeHanssen commented 1 month ago

Does this issue persits on reruns