search

deng-lab / viroprofiler

A containerized bioinformatics pipeline for viral metagenomic data analysis
https://deng-lab.github.io/viroprofiler
MIT License
23 stars 12 forks source link

Error installing Viroprofiler #15

Closed mhmism closed 1 year ago

mhmism commented 1 year ago

Description of the bug

Command 'ps' required by nextflow to collect task metrics cannot be found

Command used and terminal output

nextflow run deng-lab/viroprofiler -r main -profile singularity --mode "setup" 

N E X T F L O W  ~  version 23.04.1
Launching `https://github.com/deng-lab/viroprofiler` [sad_meninsky] DSL2 - revision: c2a1f1871b [main]

----------------------------------------------------------------------------------------------------------------
                                               __                                                              
 oooooo     oooo  o8o                        88  88                        .o88o.  o8o  oooo                   
  `888.     .8'   `"'                       88 ss 88                       888 `"  `"'  `888                   
   `888.   .8'   oooo  oooo d8b  .ooooo.     88__88    oooo d8b  .ooooo.  o888oo  oooo   888   .ooooo.  oooo d8b
    `888. .8'    `888  `888""8P d88' `88b      ||      `888""8P d88' `88b  888    `888   888  d88' `88b `888""8P
     `888.8'      888   888     888   888     _||_      888     888   888  888     888   888  888ooo888  888   
      `888'       888   888     888   888   // || \\    888     888   888  888     888   888  888    .o  888   
       `8'       o888o d888b    `Y8bod8P'  //      \\  d888b    `Y8bod8P' o888o   o888o o888o `Y8bod8P' d888b  
  ViroProfiler v0.2.4
----------------------------------------------------------------------------------------------------------------
Core Nextflow options
  revision       : main
  runName        : sad_meninsky
  containerEngine: singularity
  launchDir      : /gpfs/work5/0/gusr0570/singularity
  workDir        : /gpfs/work5/0/gusr0570/singularity/work
  projectDir     : /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler
  userName       : mibrahim30
  profile        : singularity
  configFiles    : /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/nextflow.config

Input/output options
  mode           : setup
  db             : “/gpfs/work5/0/gusr0570/viroprofiler_db/”
  outdir         : output

QC
  contamref_idx  : /home/mibrahim30/viroprofiler/contamination_refs/hg19/ref

Contig library parameters
  assemblies     : scaffolds

Others
  use_iphop      : true
  use_dram       : true

Max job request options
  max_cpus       : 1
  max_memory     : 8.GB
  max_time       : 12.h

!! Only displaying parameters that differ from the pipeline defaults !!
----------------------------------------------------------------------------------------------------------------
If you use ViroProfiler for your analysis please cite:

* The ViroProfiler pipeline
 Ru, Jinlong, et al. "ViroProfiler: a containerized bioinformatics pipeline for viral metagenomic data analysis."
 Gut Microbes 15.1 (2023): 2192522. https://doi.org/10.1080/19490976.2023.2192522

* The nf-core framework
 Ewels, Philip A., et al. "The nf-core framework for community-curated bioinformatics pipelines."
 Nature biotechnology 38.3 (2020): 276-278. https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/deng-lab/viroprofiler/blob/main/CITATIONS.md
----------------------------------------------------------------------------------------------------------------
[-        ] process > VIROPROFILER:SETUP:DB_CHECKV     -
[-        ] process > VIROPROFILER:SETUP:DB_VIBRANT    [  0%] 0 of 1
[-        ] process > VIROPROFILER:SETUP:DB_VIRSORTER2 -
[-        ] process > VIROPROFILER:SETUP:DB_DRAM       -
executor >  local (1)
[-        ] process > VIROPROFILER:SETUP:DB_CHECKV     -
[95/bc7c2b] process > VIROPROFILER:SETUP:DB_VIBRANT    [  0%] 0 of 1
[-        ] process > VIROPROFILER:SETUP:DB_VIRSORTER2 -
executor >  local (1)
[-        ] process > VIROPROFILER:SETUP:DB_CHECKV     -
[95/bc7c2b] process > VIROPROFILER:SETUP:DB_VIBRANT    [  0%] 0 of 1
[-        ] process > VIROPROFILER:SETUP:DB_VIRSORTER2 -
[-        ] process > VIROPROFILER:SETUP:DB_DRAM       -
[-        ] process > VIROPROFILER:SETUP:DB_IPHOP      -
[-        ] process > VIROPROFILER:SETUP:DB_VREFSEQ    -
Pulling Singularity image docker://denglab/viroprofiler-base:v0.2 [cache /gpfs/work5/0/gusr0570/singularity/denexecutor >  local (1)
[-        ] process > VIROPROFILER:SETUP:DB_CHECKV     -
[95/bc7c2b] process > VIROPROFILER:SETUP:DB_VIBRANT    [100%] 1 of 1, failed: 1 ✘
[-        ] process > VIROPROFILER:SETUP:DB_VIRSORTER2 -
[-        ] process > VIROPROFILER:SETUP:DB_DRAM       -
[-        ] process > VIROPROFILER:SETUP:DB_IPHOP      -
[-        ] process > VIROPROFILER:SETUP:DB_VREFSEQ    -
Pulling Singularity image docker://denglab/viroprofiler-base:v0.2 [cache /gpfs/work5/0/gusr0570/singularity/denglab-viroprofiler-base-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-virsorter2:v0.2.5 [cache /gpfs/work5/0/gusr0570/singulaexecutor >  local (1)
[-        ] process > VIROPROFILER:SETUP:DB_CHECKV     -
[95/bc7c2b] process > VIROPROFILER:SETUP:DB_VIBRANT    [100%] 1 of 1, failed: 1 ✘
[-        ] process > VIROPROFILER:SETUP:DB_VIRSORTER2 -
[-        ] process > VIROPROFILER:SETUP:DB_DRAM       -
[-        ] process > VIROPROFILER:SETUP:DB_IPHOP      -
[-        ] process > VIROPROFILER:SETUP:DB_VREFSEQ    -
Pulling Singularity image docker://denglab/viroprofiler-base:v0.2 [cache /gpfs/work5/0/gusr0570/singularity/denglab-viroprofiler-base-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-virsorter2:v0.2.5 [cache /gpfs/work5/0/gusr0570/singularity/denglab-viroprofiler-virsorter2-v0.2.5.img]
Pulling Singularity image docker://denglab/viroprofiler-geneannot:v0.2 [cache /gpfs/work5/0/gusr0570/singularity/denglab-viroprofiler-geneannot-v0.2.img]
Pulling Singularity image docker://denglab/viroprofiler-host:v0.1 [cache /gpfs/work5/0/gusr0570/singularity/denglab-viroprofiler-host-v0.1.img]
Execution cancelled -- Finishing pending tasks before exit
-[ViroProfiler] Pipeline completed with errors-
ERROR ~ Error executing process > 'VIROPROFILER:SETUP:DB_VIBRANT'

Caused by:
  Process `VIROPROFILER:SETUP:DB_VIBRANT` terminated with an error exit status (1)

Command executed:

  if [ ! -d “/gpfs/work5/0/gusr0570/viroprofiler_db/”/vibrant ]; then
      mkdir -p “/gpfs/work5/0/gusr0570/viroprofiler_db/”/vibrant
      export VIBRANT_DATA_PATH="/opt/conda/share/vibrant-1.2.1/db"
      download-db.sh “/gpfs/work5/0/gusr0570/viroprofiler_db/”/vibrant
  else
      echo "VIBRANT database already exists"
  fi

Command exit status:
  1

Command output:
  (empty)

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  Command 'ps' required by nextflow to collect task metrics cannot be found

Work dir:
  /gpfs/work5/0/gusr0570/singularity/work/95/bc7c2bae1d5487d5da1cf8c59751dc

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details

Relevant files

Aug-06 21:14:35.695 [main] DEBUG nextflow.cli.Launcher - $> nextflow run deng-lab/viroprofiler -r main -profile singularity --mode setup --db “/gpfs/work5/0/gusr0570/viroprofiler_db/” Aug-06 21:14:35.786 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.04.1 Aug-06 21:14:35.839 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/mibrahim30/.nextflow/plugins; core-plugins: nf-amazon@1.16.2,nf-azure@1.0.1,nf-codecommit@0.1.4,nf-console@1.0.5,nf-ga4gh@1.0.5,nf-google@1.7.3,nf-tower@1.5.12,nf-wave@0.8.2 Aug-06 21:14:35.858 [main] INFO org.pf4j.DefaultPluginStatusProvider - Enabled plugins: [] Aug-06 21:14:35.859 [main] INFO org.pf4j.DefaultPluginStatusProvider - Disabled plugins: [] Aug-06 21:14:35.864 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode Aug-06 21:14:35.881 [main] INFO org.pf4j.AbstractPluginManager - No plugins Aug-06 21:14:35.904 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/mibrahim30/.nextflow/scm Aug-06 21:14:37.718 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/.git/config; branch: master; remote: origin; url: https://github.com/deng-lab/viroprofiler.git Aug-06 21:14:37.741 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory] Aug-06 21:14:37.750 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/.git/config; branch: master; remote: origin; url: https://github.com/deng-lab/viroprofiler.git Aug-06 21:14:38.796 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/nextflow.config Aug-06 21:14:38.797 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/nextflow.config Aug-06 21:14:38.815 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: singularity Aug-06 21:14:39.919 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [standard, debug, test, apptainer, docker, shifter, charliecloud, test_denglab, conda, singularity, charliecloud_denglab, podman, slurm_denglab2] Aug-06 21:14:39.988 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion Aug-06 21:14:39.989 [main] INFO nextflow.cli.CmdRun - Launching https://github.com/deng-lab/viroprofiler [spontaneous_aryabhata] DSL2 - revision: c2a1f1871b [main] Aug-06 21:14:39.990 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[] Aug-06 21:14:39.990 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[] Aug-06 21:14:39.994 [main] DEBUG nextflow.secret.LocalSecretsProvider - Secrets store: /home/mibrahim30/.nextflow/secrets/store.json Aug-06 21:14:39.998 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@aa752bb] - activable => nextflow.secret.LocalSecretsProvider@aa752bb Aug-06 21:14:40.070 [main] DEBUG nextflow.Session - Session UUID: c77b1232-c098-4759-b0be-8111b5210aac Aug-06 21:14:40.071 [main] DEBUG nextflow.Session - Run name: spontaneous_aryabhata Aug-06 21:14:40.071 [main] DEBUG nextflow.Session - Executor pool size: 4 Aug-06 21:14:40.088 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=12; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false Aug-06 21:14:40.132 [main] DEBUG nextflow.cli.CmdRun - Version: 23.04.1 build 5866 Created: 15-04-2023 06:51 UTC (08:51 CEST) System: Linux 4.18.0-372.57.1.el8_6.x86_64 Runtime: Groovy 3.0.16 on OpenJDK 64-Bit Server VM 20-internal-adhoc..src Encoding: UTF-8 (UTF-8) Process: 2168861@int6 [172.18.63.192] CPUs: 4 - Mem: 16 GB (15.7 GB) - Swap: 0 (0) Aug-06 21:14:40.186 [main] DEBUG nextflow.Session - Work-dir: /gpfs/work5/0/gusr0570/work [gpfs] Aug-06 21:14:40.229 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[] Aug-06 21:14:40.241 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory Aug-06 21:14:40.416 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory Aug-06 21:14:40.429 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 5; maxThreads: 1000 Aug-06 21:14:40.592 [main] DEBUG nextflow.Session - Session start Aug-06 21:14:40.598 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /gpfs/work5/0/gusr0570/output/pipeline_info/execution_trace_2023-08-06_21-14-39.txt Aug-06 21:14:40.643 [main] DEBUG nextflow.Session - Using default localLib path: /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/lib Aug-06 21:14:40.656 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/lib Aug-06 21:14:40.657 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/lib/nfcore_external_java_deps.jar Aug-06 21:14:41.726 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution Aug-06 21:14:41.985 [main] INFO nextflow.Nextflow -

----------------------------------------------------------------------------------------------------------------    oooooo oooo o8o 88 88 .o88o. o8o oooo   888. .8'"' 88 ss 88 888 ""' 888  888. .8' oooo oooo d8b .ooooo. 8888 oooo d8b .ooooo. o888oo oooo 888 .ooooo. oooo d8b   888. .8'888 888""8P d88'88b || 888""8P d88'88b 888 888 888 d88'88b 888""8P  888.8' 888 888 888 888 || 888 888 888 888 888 888 888ooo888 888   888' 888 888 888 888 // || \\ 888 888 888 888 888 888 888 .o 888  8' o888o d888b Y8bod8P' // \\ d888bY8bod8P' o888o o888o o888o `Y8bod8P' d888b   ViroProfiler v0.2.4 ---------------------------------------------------------------------------------------------------------------- Core Nextflow options revision : main runName : spontaneous_aryabhata containerEngine: singularity launchDir : /gpfs/work5/0/gusr0570 workDir : /gpfs/work5/0/gusr0570/work projectDir : /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler userName : mibrahim30 profile : singularity configFiles : /home/mibrahim30/.nextflow/assets/deng-lab/viroprofiler/nextflow.config

Input/output options mode : setup db : “/gpfs/work5/0/gusr0570/viroprofiler_db/” outdir : output

QC contamref_idx : /home/mibrahim30/viroprofiler/contamination_refs/hg19/ref

Contig library parameters assemblies : scaffolds

Others use_iphop : true use_dram : true

Max job request options max_cpus : 1 max_memory : 8.GB max_time : 12.h

!! Only displaying parameters that differ from the pipeline defaults !! ---------------------------------------------------------------------------------------------------------------- If you use ViroProfiler for your analysis please cite:

Caused by: Process VIROPROFILER:SETUP:DB_VIBRANT terminated with an error exit status (1)

Command executed:

if [ ! -d “/gpfs/work5/0/gusr0570/viroprofiler_db/”/vibrant ]; then mkdir -p “/gpfs/work5/0/gusr0570/viroprofiler_db/”/vibrant export VIBRANT_DATA_PATH="/opt/conda/share/vibrant-1.2.1/db" download-db.sh “/gpfs/work5/0/gusr0570/viroprofiler_db/”/vibrant else echo "VIBRANT database already exists" fi

Command exit status: 1

Command output: (empty)

Command error: INFO: Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred INFO: Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred Command 'ps' required by nextflow to collect task metrics cannot be found

Work dir: /gpfs/work5/0/gusr0570/work/e6/3a2b5b4f8bc4fe1b9844bd60cd24b0

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line Aug-06 21:14:48.516 [Task monitor] INFO nextflow.Session - Execution cancelled -- Finishing pending tasks before exit Aug-06 21:14:48.520 [main] DEBUG nextflow.Session - Session await > all processes finished Aug-06 21:14:48.520 [main] DEBUG nextflow.Session - Session await > all barriers passed Aug-06 21:14:48.524 [Actor Thread 1] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=VIROPROFILER:SETUP:DB_IPHOP; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Aug-06 21:14:48.524 [Actor Thread 2] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=VIROPROFILER:SETUP:DB_CHECKV; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Aug-06 21:14:48.524 [Actor Thread 6] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=VIROPROFILER:SETUP:DB_VREFSEQ; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Aug-06 21:14:48.524 [Actor Thread 4] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=VIROPROFILER:SETUP:DB_VIRSORTER2; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Aug-06 21:14:48.524 [Actor Thread 5] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=VIROPROFILER:SETUP:DB_DRAM; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Aug-06 21:14:48.528 [main] INFO nextflow.Nextflow - -[ViroProfiler] Pipeline completed with errors- Aug-06 21:14:48.534 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=839ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=4 GB; ] Aug-06 21:14:48.534 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file Aug-06 21:14:48.535 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report Aug-06 21:14:48.563 [main] DEBUG nextflow.trace.ReportObserver - Execution report summary data: [{"cpuUsage":null,"process":"DB_VIBRANT","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":null,"cpu":null,"time":{"mean":839,"min":839,"q1":839,"q2":839,"q3":839,"max":839,"minLabel":"VIROPROFILER:SETUP:DB_VIBRANT","maxLabel":"VIROPROFILER:SETUP:DB_VIBRANT","q1Label":"VIROPROFILER:SETUP:DB_VIBRANT","q2Label":"VIROPROFILER:SETUP:DB_VIBRANT","q3Label":"VIROPROFILER:SETUP:DB_VIBRANT"},"writes":null}] Aug-06 21:14:49.569 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline Aug-06 21:14:49.744 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done Aug-06 21:14:49.821 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'FileTransfer' shutdown completed (hard=false) Aug-06 21:14:49.822 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

System information

Nextflow version version 23.04.1 Hadware HPC

mhmism commented 1 year ago

This specific error was due to an old version of singularity. Therefore, I am closing this issue.