search

epi2me-labs / wf-human-variation

Other
103 stars 45 forks source link

Error: An unexpected error occurred while trying to open file snpEff.jar #216

Open NorikazuHARA opened 2 months ago

NorikazuHARA commented 2 months ago

Operating System

CentOS 7

Other Linux

No response

Workflow Version

v2.3.1-g6a6daa6

Workflow Execution

Command line (Local)

Other workflow execution

No response

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-human-variation \ --bam 'wf-human-variation-demo/demo.bam' \ --ref 'wf-human-variation-demo/demo.fasta' \ --bed 'wf-human-variation-demo/demo.bed' \ --sample_name 'DEMO' \ --snp \ --sv \ --mod \ --phased \ -profile standard .nextflow.log

Workflow Execution - CLI Execution Profile

None

What happened?

I ran the wf-human-variation workflow for the first time on the demo data on my local server. During annotation with snpEff, I encountered an error:

"Error: An unexpected error occurred while trying to open file /opt/custflow/epi2meuser/conda/share/snpeff-5.1-2/snpEff.jar"

Relevant log output

Sep-06 14:19:24.595 [main] DEBUG nextflow.cli.Launcher - $> nextflow run epi2me-labs/wf-human-variation --bam wf-human-variation-demo/demo.bam --ref wf-human-variation-demo/demo.fasta --bed wf-human-variation-demo/demo.bed --sample_name DEMO --snp --sv --mod --phased -profile standard
Sep-06 14:19:24.678 [main] DEBUG nextflow.cli.CmdRun - N E X T F L O W  ~  version 24.04.4
Sep-06 14:19:24.705 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/hara/.nextflow/plugins; core-plugins: nf-amazon@2.5.3,nf-azure@1.6.1,nf-cloudcache@0.4.1,nf-codecommit@0.2.1,nf-console@1.1.3,nf-ga4gh@1.3.0,nf-google@1.13.2-patch1,nf-tower@1.9.1,nf-wave@1.4.2-patch1
Sep-06 14:19:24.719 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Sep-06 14:19:24.720 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Sep-06 14:19:24.724 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.12.0 in 'deployment' mode
Sep-06 14:19:24.739 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Sep-06 14:19:24.756 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/hara/.nextflow/scm
Sep-06 14:19:25.485 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-human-variation.git
Sep-06 14:19:25.547 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Sep-06 14:19:25.560 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-human-variation.git
Sep-06 14:19:26.394 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/nextflow.config
Sep-06 14:19:26.396 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/nextflow.config
Sep-06 14:19:26.404 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/hara/.nextflow/secrets/store.json
Sep-06 14:19:26.407 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@477021ee] - activable => nextflow.secret.LocalSecretsProvider@477021ee
Sep-06 14:19:26.418 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Sep-06 14:19:26.799 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declaration
Sep-06 14:19:26.816 [main] DEBUG nextflow.cli.CmdRun - Launching `https://github.com/epi2me-labs/wf-human-variation` [backstabbing_bell] DSL2 - revision: 6a6daa65aa [master]
Sep-06 14:19:26.818 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Sep-06 14:19:26.818 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Sep-06 14:19:26.903 [main] DEBUG nextflow.Session - Session UUID: 840bf982-026e-4c74-8c1e-964a364c6c51
Sep-06 14:19:26.904 [main] DEBUG nextflow.Session - Run name: backstabbing_bell
Sep-06 14:19:26.904 [main] DEBUG nextflow.Session - Executor pool size: 96
Sep-06 14:19:26.914 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Sep-06 14:19:26.920 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=288; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Sep-06 14:19:26.942 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 24.04.4 build 5917
  Created: 01-08-2024 07:05 UTC (16:05 JDT)
  System: Linux 3.10.0-957.27.2.el7.x86_64
  Runtime: Groovy 4.0.21 on OpenJDK 64-Bit Server VM 11.0.13+7-b1751.21
  Encoding: UTF-8 (ANSI_X3.4-1968)
  Process: 285209@DPeR840 [192.168.12.110]
  CPUs: 96 - Mem: 376.2 GB (60 GB) - Swap: 32 GB (32 GB)
Sep-06 14:19:26.962 [main] DEBUG nextflow.Session - Work-dir: /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work [nfs]
Sep-06 14:19:27.032 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Sep-06 14:19:27.063 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Sep-06 14:19:27.180 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Sep-06 14:19:27.200 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 97; maxThreads: 1000
Sep-06 14:19:27.349 [main] DEBUG nextflow.Session - Session start
Sep-06 14:19:27.353 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/[:]/execution/trace.txt
Sep-06 14:19:27.377 [main] DEBUG nextflow.Session - Using default localLib path: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/lib
Sep-06 14:19:27.381 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/lib
Sep-06 14:19:27.382 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/lib/nfcore_external_java_deps.jar
Sep-06 14:19:29.749 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Sep-06 14:19:39.906 [main] INFO  nextflow.Nextflow - 
||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-human-variation v2.3.1-g6a6daa6
--------------------------------------------------------------------------------
Core Nextflow options
  revision       : master
  runName        : backstabbing_bell
  containerEngine: docker
  container      : [withLabel:wf_human_snp:ontresearch/wf-human-variation-snp:sha17e686336bf6305f9c90b36bc52ff9dd1fa73ee9, withLabel:wf_human_sv:ontresearch/wf-human-variation-sv:shac591518dd32ecc3936666c95ff08f6d7474e9728, withLabel:wf_human_mod:ontresearch/modkit:shae137452eb41f5f12b790774cafe15bc97f48d4d0, withLabel:wf_cnv:ontresearch/wf-cnv:sha428cb19e51370020ccf29ec2af4eead44c6a17c2, withLabel:wf_human_str:ontresearch/wf-human-variation-str:shadd2f2963fe39351d4e0d6fa3ca54e1064c6ec057, withLabel:snpeff_annotation:ontresearch/snpeff:shadcc812849019640d4d2939703fbb8777256e41ad, withLabel:wf_common:ontresearch/wf-common:shad399cf22079b5b153920ac39ee40095a677933f1, withLabel:spectre:ontresearch/spectre:sha49a9fe474da9860f84f08f17f137b47a010b1834, default:ontresearch/wf-human-variation:sha2b856c1f358ddf1576217a336bc0e9864b6dc0ed]
  launchDir      : /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp
  workDir        : /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work
  projectDir     : /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation
  userName       : hara
  profile        : standard
  configFiles    : /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/nextflow.config

Workflow Options
  sv             : true
  snp            : true
  mod            : true

Main options
  sample_name    : DEMO
  bam            : wf-human-variation-demo/demo.bam
  ref            : wf-human-variation-demo/demo.fasta
  bed            : wf-human-variation-demo/demo.bed
  phased         : true

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-human-variation for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-human-variation v2.3.1-g6a6daa6.
--------------------------------------------------------------------------------
Sep-06 14:19:40.054 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `wf_common` for process with name prepare_reference:cram_cache
Sep-06 14:19:40.068 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Sep-06 14:19:40.068 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Sep-06 14:19:40.075 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Sep-06 14:19:40.082 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=96; memory=376.2 GB; capacity=96; pollInterval=100ms; dumpInterval=5m
Sep-06 14:19:40.085 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)

~~~~~
~~~~~

Sep-06 14:22:49.503 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Sep-06 14:22:49.503 [Task submitter] INFO  nextflow.Session - [94/0a2970] Submitted process > refine_with_sv (1)
Sep-06 14:22:49.508 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Sep-06 14:22:49.509 [Task submitter] INFO  nextflow.Session - [86/eea2ff] Submitted process > sv:annotate_sv_vcf (1)
Sep-06 14:22:50.538 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 121; name: sv:annotate_sv_vcf (1); status: COMPLETED; exit: 1; error: -; workDir: /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work/86/eea2ffaf582140b7daf89a6933b9f0]
Sep-06 14:22:50.542 [TaskFinalizer-8] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=sv:annotate_sv_vcf (1); work-dir=/disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work/86/eea2ffaf582140b7daf89a6933b9f0
  error [nextflow.exception.ProcessFailedException]: Process `sv:annotate_sv_vcf (1)` terminated with an error exit status (1)
Sep-06 14:22:50.564 [TaskFinalizer-8] ERROR nextflow.processor.TaskProcessor - Error executing process > 'sv:annotate_sv_vcf (1)'

Caused by:
  Process `sv:annotate_sv_vcf (1)` terminated with an error exit status (1)

Command executed:

  if [ "*" == '*' ]; then
      # SV is quick to annotate, dont bother breaking it apart
      INPUT_FILENAME=input.vcf.gz
      OUTPUT_LABEL="sv"
  else
      # SNP is slow to annotate, we'll break it apart by contig
      # and merge it back later. filter the master VCF to current contig
      # Also, relabel chrM as chrMT so SnpEff can annotate
      bcftools view -r * input.vcf.gz | sed 's/^chrM\t/chrMT\t/' | bgzip > input.chr.vcf.gz

      INPUT_FILENAME=input.chr.vcf.gz
      OUTPUT_LABEL="sv.*"
  fi

  # deal with samples which aren't hg19 or hg38
  if [[ "hg38" != "hg38" ]] && [[ "hg38" != "hg19" ]]; then
      # return the original VCF and index as the outputs
      cp ${INPUT_FILENAME} DEMO.wf_${OUTPUT_LABEL}.vcf.gz
      cp ${INPUT_FILENAME}.tbi DEMO.wf_${OUTPUT_LABEL}.vcf.gz.tbi
  else
      # do some annotation
      if [[ "hg38" == "hg38" ]]; then
          snpeff_db="GRCh38.p13"
          clinvar_vcf="${CLINVAR_PATH}/clinvar_GRCh38.vcf.gz"

      elif [[ "hg38" == "hg19" ]]; then
          snpeff_db="GRCh37.p13"
          clinvar_vcf="${CLINVAR_PATH}/clinvar_GRCh37.vcf.gz"
      fi

      # Revert back chrMT to chrM
      snpEff -Xmx5g ann -noStats -noLog $snpeff_db ${INPUT_FILENAME} | sed 's/^chrMT\t/chrM\t/' > DEMO.intermediate.snpeff_annotated.vcf

      # Add ClinVar annotations
      SnpSift annotate $clinvar_vcf DEMO.intermediate.snpeff_annotated.vcf | bgzip > DEMO.wf_${OUTPUT_LABEL}.vcf.gz
      tabix DEMO.wf_${OUTPUT_LABEL}.vcf.gz

      # tidy up
      rm DEMO.intermediate*
  fi

Command exit status:
  1

Command output:
  (empty)

Command error:
  Picked up JAVA_TOOL_OPTIONS: -Xlog:disable -Xlog:all=warning:stderr
  Error: An unexpected error occurred while trying to open file /opt/custflow/epi2meuser/conda/share/snpeff-5.1-2/snpEff.jar

Work dir:
  /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work/86/eea2ffaf582140b7daf89a6933b9f0

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
Sep-06 14:22:50.569 [TaskFinalizer-8] DEBUG nextflow.Session - Session aborted -- Cause: Process `sv:annotate_sv_vcf (1)` terminated with an error exit status (1)
Sep-06 14:22:50.601 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Sep-06 14:22:50.607 [TaskFinalizer-8] DEBUG nextflow.Session - The following nodes are still active:
[process] concat_refined_snp
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: output_label
  port 2: (cntrl) -     ; channel: $

[process] bed_filter
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (queue) OPEN  ; channel: bed
  port 2: (value) bound ; channel: subworkflow
  port 3: (value) bound ; channel: file_type
  port 4: (cntrl) -     ; channel: $

[process] annotate_snp_vcf
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: genome
  port 2: (value) bound ; channel: output_label
  port 3: (cntrl) -     ; channel: $

[process] concat_snp_vcfs
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: output_label
  port 2: (cntrl) -     ; channel: $

[process] sift_clinvar_snp_vcf
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (queue) OPEN  ; channel: genome
  port 2: (value) bound ; channel: output_label
  port 3: (cntrl) -     ; channel: $

[process] vcfStats
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

[process] report_snp:makeReport
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: versions
  port 2: (value) bound ; channel: params.json
  port 3: (queue) OPEN  ; channel: clinvar_vcf
  port 4: (cntrl) -     ; channel: $

[process] output_snp
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: fname
  port 1: (cntrl) -     ; channel: $

[process] combine_metrics_json
  status=ACTIVE
  port 0: (value) OPEN  ; channel: jsons
  port 1: (queue) OPEN  ; channel: flagstat.tsv
  port 2: (queue) OPEN  ; channel: hists
  port 3: (queue) OPEN  ; channel: -
  port 4: (queue) OPEN  ; channel: mosdepth.summary.txt
  port 5: (queue) OPEN  ; channel: haplocheck
  port 6: (queue) OPEN  ; channel: sex
  port 7: (cntrl) -     ; channel: $

[process] publish_artifact
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: fname
  port 1: (cntrl) -     ; channel: $

Sep-06 14:22:51.475 [main] DEBUG nextflow.Session - Session await > all processes finished
Sep-06 14:22:51.475 [main] DEBUG nextflow.Session - Session await > all barriers passed
Sep-06 14:22:51.806 [main] WARN  n.processor.TaskPollingMonitor - Killing running tasks (3)
Sep-06 14:22:51.819 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill refine_with_sv (3) -- command: kill -TERM 372743; exit: 1 
 bash: line 0: kill: (372743) - No such process

Sep-06 14:22:51.824 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill refine_with_sv (2) -- command: kill -TERM 372750; exit: 1 
 bash: line 0: kill: (372750) - No such process

Sep-06 14:22:51.827 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill refine_with_sv (1) -- command: kill -TERM 372762; exit: 1 
 bash: line 0: kill: (372762) - No such process

Sep-06 14:22:51.829 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=117; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=1; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=3; succeedDuration=34m 59s; failedDuration=962ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=51; peakCpus=54; peakMemory=216 GB; ]
Sep-06 14:22:51.830 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Sep-06 14:22:51.831 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Sep-06 14:22:52.825 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Sep-06 14:22:52.922 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Sep-06 14:22:52.952 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

no

Other demo data information

No response

vlshesketh commented 1 month ago

Hi @NorikazuHARA, apologies with the delay in getting back to you about this. I'm afraid I've been unable to reproduce this error locally - the workflow has been updated to v2.4.1, please can you try again with the latest version by doing:

nextflow pull epi2me-labs/wf-human-variation

and let us know if you're still getting the same error?

NorikazuHARA commented 1 month ago

@vlshesketh Thanks for your reply.

I updated the workflow to v2.4.1 and tried with it. But I got the same error during annotation with snpEff.

The other tasks seem to work fine. So I'll try to annotate vcf files manually instead.

Anyway, thanks a lot!