search

epi2me-labs / wf-transcriptomes

Other
71 stars 31 forks source link

[Bug]: wf-glue: error: unrecognized arguments: batches in Clustering #17

Closed Johnsonzcode closed 1 year ago

Johnsonzcode commented 1 year ago

What happened?

wf-glue: error: unrecognized arguments: batches in denovo assembly Clustering

(nextflow) [poultrylab1@pbsnode01 ONT]$ bash wf-transcriptome.sh
N E X T F L O W  ~  version 23.04.0
Launching `/storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/main.nf` [high_pauling] DSL2 - revision: 0ca5d89e65

WARN: Found unexpected parameters:
* --version: false
- Ignore this warning: params.schema_ignore_params = "version"

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-transcriptomes v0.1.10
--------------------------------------------------------------------------------
Core Nextflow options
  runName                : high_pauling
  containerEngine        : docker
  launchDir              : /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT
  workDir                : /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/male_workspace_dir
  projectDir             : /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10
  userName               : poultrylab1
  profile                : standard
  configFiles            : /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/nextflow.config

Input Options
  fastq                  : male_ONT_full_length.fastq.gz
  transcriptome_source   : denovo
  ref_transcriptome      : null

Output Options
  out_dir                : male_outdir

Sample Options
  sample                 : male

Options for reference-based workflow
  plot_gffcmp_stats      : true

Options for de novo-based workflow
  isOnClust2_sort_options: --batch-size -1 --kmer-size 11 --window-size 15 --min-shared 5 --min-qual 7.0 --mapped-threshold 0.65 --aligned-threshold 0.2 --min-fraction 0.8 --min-prob-no-hits 0.0 -M -1 -P 50
0 -g 50 -c 150 -F 2
Differential Expression Options
  condition_sheet        : test_data/condition_sheet.tsv

Advanced Options
  threads                : 4
  bundle_min_reads       : 50000

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-transcriptomes for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-transcriptomes v0.1.10.
--------------------------------------------------------------------------------
Checking fastq input.
executor >  local (8)
[72/3705a7] process > fastcat (1)                                  [100%] 1 of 1 ✔
[cb/6eeac2] process > pipeline:collectFastqIngressResultsInDir (1) [100%] 1 of 1 ✔
executor >  local (8)
[72/3705a7] process > fastcat (1)                                  [100%] 1 of 1 ✔
[cb/6eeac2] process > pipeline:collectFastqIngressResultsInDir (1) [100%] 1 of 1 ✔
executor >  local (8)
[72/3705a7] process > fastcat (1)                                  [100%] 1 of 1 ✔
[cb/6eeac2] process > pipeline:collectFastqIngressResultsInDir (1) [100%] 1 of 1 ✔
[0e/a1f30c] process > pipeline:getVersions                         [100%] 1 of 1 ✔
[1f/c4d5f0] process > pipeline:getParams                           [100%] 1 of 1 ✔
[4e/a46c9c] process > pipeline:preprocess_reads (1)                [100%] 1 of 1 ✔
[fe/48ea3b] process > pipeline:denovo_assembly:make_batches (1)    [100%] 1 of 1 ✔
[e2/f907a8] process > pipeline:denovo_assembly:clustering (1)      [100%] 1 of 1, failed: 1 ✘
[-        ] process > pipeline:denovo_assembly:dump_clusters       -
[-        ] process > pipeline:denovo_assembly:build_backbones     -
[-        ] process > pipeline:denovo_assembly:merge_cds           -
[-        ] process > pipeline:denovo_assembly:cds_align           -
[-        ] process > pipeline:split_bam                           -
[-        ] process > pipeline:assemble_transcripts                -
[-        ] process > pipeline:merge_gff_bundles                   -
[-        ] process > pipeline:run_gffcompare                      -
[-        ] process > pipeline:get_transcriptome                   -
[-        ] process > pipeline:makeReport                          -
[74/fbd298] process > output (1)                                   [100%] 1 of 1 ✔
ERROR ~ Error executing process > 'pipeline:denovo_assembly:clustering (1)'

Caused by:
  Process `pipeline:denovo_assembly:clustering (1)` terminated with an error exit status (2)

Command executed:

  workflow-glue run_isonclust2 batches

Command exit status:
  2

Command output:
  (empty)

Command error:
  /home/epi2melabs/conda/lib/python3.8/site-packages/gffutils/parser.py:19: DeprecationWarning: invalid escape sequence \w
    gff3_kw_pat = re.compile('\w+=')
  usage: wf-glue [-h] [--debug | --quiet] [-v]
                 {check_sample_sheet,compute_cluster_quality,generate_pychopper_stats,generate_tracking_summary,merge_count_tsvs,report,run_isonclust2}
                 ...
  wf-glue: error: unrecognized arguments: batches

Work dir:
  /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/male_workspace_dir/e2/f907a844c6751a3b67ae862bb704d1

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details
(nextflow) [poultrylab1@pbsnode01 ONT]$ cat /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ab/c4ddcbe7604006308a3b620f2c67c1/.command.log
/home/epi2melabs/conda/lib/python3.8/site-packages/gffutils/parser.py:19: DeprecationWarning: invalid escape sequence \w
  gff3_kw_pat = re.compile('\w+=')
usage: wf-glue [-h] [--debug | --quiet] [-v]
               {check_sample_sheet,compute_cluster_quality,generate_pychopper_stats,generate_tracking_summary,merge_count_tsvs,report,run_isonclust2}
               ...
wf-glue: error: unrecognized arguments: batches
(nextflow) [poultrylab1@pbsnode01 ONT]$ cat /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ab/c4ddcbe7604006308a3b620f2c67c1/.command.sh
#!/bin/bash -euo pipefail
workflow-glue run_isonclust2 batches

My servers is Centos 7, using github branches version and nextflow pulling version. Same error.

Operating System

ubuntu 20.04

Workflow Execution

Command line

Workflow Execution - EPI2ME Labs Versions

Using command line not EPI2ME Labs.

Workflow Execution - CLI Execution Profile

Docker

Workflow Version

wf-transcriptomes v0.1.10

Relevant log output

(nextflow) [poultrylab1@pbsnode01 ONT]$ le .nextflow.log
Apr-11 20:01:45.216 [main] DEBUG nextflow.cli.Launcher - $> nextflow run /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/main.nf --fastq female_ONT_full_length.fastq.gz --transcriptome_source denovo --out_dir female_outdir -w female_workspace_dir --sample female
Apr-11 20:01:45.360 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.04.0
Apr-11 20:01:45.396 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/storage-01/poultrylab1/.nextflow/plugins; core-plugins: nf-amazon@1.16.1,nf-azure@1.0.0,nf-codecommit@0.1.3,nf-console@1.0.5,nf-ga4gh@1.0.4,nf-google@1.7.2,nf-tower@1.5.11,nf-wave@0.8.1
Apr-11 20:01:45.812 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Apr-11 20:01:45.814 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Apr-11 20:01:45.821 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Apr-11 20:01:45.843 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Apr-11 20:01:45.877 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/nextflow.config
Apr-11 20:01:45.881 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/nextflow.config
Apr-11 20:01:45.950 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Apr-11 20:01:47.245 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Apr-11 20:01:47.268 [main] INFO  nextflow.cli.CmdRun - Launching `/storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/main.nf` [goofy_koch] DSL2 - revision: 0ca5d89e65
Apr-11 20:01:47.269 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Apr-11 20:01:47.269 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Apr-11 20:01:47.280 [main] DEBUG nextflow.secret.LocalSecretsProvider - Secrets store: /storage-01/poultrylab1/.nextflow/secrets/store.json
Apr-11 20:01:47.284 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@5c134052] - activable => nextflow.secret.LocalSecretsProvider@5c134052
Apr-11 20:01:47.355 [main] DEBUG nextflow.Session - Session UUID: e1d73dac-0326-4c9d-8bac-b263438ad622
Apr-11 20:01:47.355 [main] DEBUG nextflow.Session - Run name: goofy_koch
Apr-11 20:01:47.356 [main] DEBUG nextflow.Session - Executor pool size: 208
Apr-11 20:01:47.375 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=624; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Apr-11 20:01:47.410 [main] DEBUG nextflow.cli.CmdRun -
  Version: 23.04.0 build 5857
  Created: 01-04-2023 21:09 UTC (02-04-2023 05:09 CDT)
  System: Linux 3.10.0-1127.19.1.el7.x86_64
  Runtime: Groovy 3.0.16 on OpenJDK 64-Bit Server VM 11.0.13+7-b1751.21
  Encoding: UTF-8 (UTF-8)
  Process: 89827@pbsnode01 [127.0.0.1]
  CPUs: 208 - Mem: 1007.1 GB (219.5 GB) - Swap: 64 GB (42.6 GB)
Apr-11 20:01:47.515 [main] DEBUG nextflow.Session - Work-dir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir [xfs]
Apr-11 20:01:47.536 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Apr-11 20:01:47.557 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Apr-11 20:01:47.654 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Apr-11 20:01:47.668 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 209; maxThreads: 1000
Apr-11 20:01:47.752 [main] DEBUG nextflow.Session - Session start
Apr-11 20:01:47.757 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_outdir/execution/trace.txt
Apr-11 20:01:47.770 [main] DEBUG nextflow.Session - Using default localLib path: /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/lib
Apr-11 20:01:47.773 [main] DEBUG nextflow.Session - Adding to the classpath library: /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/lib
Apr-11 20:01:47.774 [main] DEBUG nextflow.Session - Adding to the classpath library: /storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/lib/nfcore_external_java_deps.jar
Apr-11 20:01:50.647 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Apr-11 20:01:52.234 [main] WARN  nextflow.Nextflow - Found unexpected parameters:
* --version: false
Apr-11 20:01:52.236 [main] INFO  nextflow.Nextflow - - ESC[2mIgnore this warning: params.schema_ignore_params = "version" ESC[0m
Apr-11 20:01:52.262 [main] INFO  nextflow.Nextflow -
ESC[0;92m||||||||||   ESC[0mESC[2m_____ ____ ___ ____  __  __ _____      _       _
ESC[0;92m||||||||||  ESC[0mESC[2m| ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
ESC[0;33m|||||       ESC[0mESC[2m|  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
ESC[0;33m|||||       ESC[0mESC[2m| |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
ESC[0;94m||||||||||  ESC[0mESC[2m|_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
ESC[0;94m||||||||||  ESC[0mESC[1mwf-transcriptomes v0.1.10ESC[0m
ESC[2m--------------------------------------------------------------------------------ESC[0m
ESC[1mCore Nextflow optionsESC[0m
  ESC[0;34mrunName                : ESC[0;32mgoofy_kochESC[0m
  ESC[0;34mcontainerEngine        : ESC[0;32mdockerESC[0m
  ESC[0;34mlaunchDir              : ESC[0;32m/storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONTESC[0m
  ESC[0;34mworkDir                : ESC[0;32m/storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dirESC[0m
  ESC[0;34mprojectDir             : ESC[0;32m/storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10ESC[0m
  ESC[0;34muserName               : ESC[0;32mpoultrylab1ESC[0m
  ESC[0;34mprofile                : ESC[0;32mstandardESC[0m
  ESC[0;34mconfigFiles            : ESC[0;32m/storage-02/zhaoqiangsen/software/wf-transcriptomes-0.1.10/nextflow.configESC[0m

ESC[1mInput OptionsESC[0m
  ESC[0;34mfastq                  : ESC[0;32mfemale_ONT_full_length.fastq.gzESC[0m
  ESC[0;34mtranscriptome_source   : ESC[0;32mdenovoESC[0m
  ESC[0;34mref_transcriptome      : ESC[0;32mnullESC[0m

ESC[1mOutput OptionsESC[0m
  ESC[0;34mout_dir                : ESC[0;32mfemale_outdirESC[0m

ESC[1mSample OptionsESC[0m
  ESC[0;34msample                 : ESC[0;32mfemaleESC[0m

ESC[1mOptions for reference-based workflowESC[0m
  ESC[0;34mplot_gffcmp_stats      : ESC[0;32mtrueESC[0m

ESC[1mOptions for de novo-based workflowESC[0m
  ESC[0;34misOnClust2_sort_options: ESC[0;32m--batch-size -1 --kmer-size 11 --window-size 15 --min-shared 5 --min-qual 7.0 --mapped-threshold 0.65 --aligned-threshold 0.2 --min-fraction 0.8 --min-prob-no-hits 0.0 -M -1 -P 500 -g 50 -c 150 -F 2ESC[0m

ESC[1mDifferential Expression OptionsESC[0m
  ESC[0;34mcondition_sheet        : ESC[0;32mtest_data/condition_sheet.tsvESC[0m

ESC[1mAdvanced OptionsESC[0m
  ESC[0;34mthreads                : ESC[0;32m4ESC[0m
  ESC[0;34mbundle_min_reads       : ESC[0;32m50000ESC[0m

!! Only displaying parameters that differ from the pipeline defaults !!
ESC[2m--------------------------------------------------------------------------------ESC[0m
If you use epi2me-labs/wf-transcriptomes for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

ESC[2m--------------------------------------------------------------------------------ESC[0m
This is epi2me-labs/wf-transcriptomes v0.1.10.
ESC[2m--------------------------------------------------------------------------------ESC[0m
Apr-11 20:01:55.668 [main] INFO  nextflow.Nextflow - Checking fastq input.
Apr-11 20:01:55.924 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name fastcat
Apr-11 20:01:55.933 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:55.933 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:55.944 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Apr-11 20:01:55.953 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=4; memory=8 GB; capacity=208; pollInterval=100ms; dumpInterval=5m
Apr-11 20:01:56.184 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:collectFastqIngressResultsInDir
Apr-11 20:01:56.188 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.189 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.197 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:getVersions
Apr-11 20:01:56.198 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.198 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.200 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:getParams
Apr-11 20:01:56.201 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.201 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.239 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:preprocess_reads
Apr-11 20:01:56.245 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.245 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.276 [main] INFO  nextflow.Nextflow - Doing de novo assembly
Apr-11 20:01:56.280 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:denovo_assembly:make_batches
Apr-11 20:01:56.282 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.282 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.287 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:denovo_assembly:clustering
Apr-11 20:01:56.287 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.288 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.303 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:denovo_assembly:dump_clusters
Apr-11 20:01:56.304 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.304 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.320 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:denovo_assembly:build_backbones
Apr-11 20:01:56.321 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.322 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.332 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:denovo_assembly:merge_cds
Apr-11 20:01:56.333 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.333 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.342 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:denovo_assembly:cds_align
Apr-11 20:01:56.343 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.343 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.362 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:split_bam
Apr-11 20:01:56.363 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.363 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.370 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:assemble_transcripts
Apr-11 20:01:56.370 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.370 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.377 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:merge_gff_bundles
Apr-11 20:01:56.377 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.377 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.384 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:run_gffcompare
Apr-11 20:01:56.385 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.385 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.402 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:get_transcriptome
Apr-11 20:01:56.403 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.403 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.427 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name pipeline:makeReport
Apr-11 20:01:56.428 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.428 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.460 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:isoforms` matches labels `isoforms` for process with name output
Apr-11 20:01:56.461 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-11 20:01:56.461 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-11 20:01:56.466 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: mergeTPM, pipeline:getParams, pipeline:get_transcriptome, merge_gff_bundles, validate_sample_sheet, pipeline:assemble_transcripts, deAnalysis, cds_align, build_backbones, dump_clusters, jaffal, pipeline:denovo_assembly:cds_align, pipeline:makeReport, output, pipeline:split_bam, make_batches, pipeline:denovo_assembly:dump_clusters, pipeline:denovo_assembly:clustering, merge_cds, collectFastqIngressResultsInDir, map_transcriptome, makeReport, pipeline:merge_gff_bundles, build_minimap_index, pipeline:denovo_assembly:merge_cds, getVersions, assemble_transcripts, run_gffcompare, get_transcriptome, merge_transcriptomes, preprocess_reads, pipeline:collectFastqIngressResultsInDir, count_transcripts, pipeline:denovo_assembly:make_batches, pipeline:getVersions, pipeline:preprocess_reads, move_or_compress, cluster_quality, mergeCounts, fastcat, plotResults, split_bam, build_minimap_index_transcriptome, clustering, map_reads, pipeline:denovo_assembly:build_backbones, pipeline:run_gffcompare, getParams
Apr-11 20:01:56.471 [main] DEBUG nextflow.Session - Igniting dataflow network (22)
Apr-11 20:01:56.472 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > fastcat
Apr-11 20:01:56.474 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:collectFastqIngressResultsInDir
Apr-11 20:01:56.475 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:getVersions
Apr-11 20:01:56.477 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:getParams
Apr-11 20:01:56.495 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess_reads
Apr-11 20:01:56.497 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:denovo_assembly:make_batches
Apr-11 20:01:56.497 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:denovo_assembly:clustering
Apr-11 20:01:56.498 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:denovo_assembly:dump_clusters
Apr-11 20:01:56.498 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:denovo_assembly:build_backbones
Apr-11 20:01:56.499 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:denovo_assembly:merge_cds
Apr-11 20:01:56.499 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:denovo_assembly:cds_align
Apr-11 20:01:56.500 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:split_bam
Apr-11 20:01:56.500 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:assemble_transcripts
Apr-11 20:01:56.501 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:merge_gff_bundles
Apr-11 20:01:56.501 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:run_gffcompare
Apr-11 20:01:56.502 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:get_transcriptome
Apr-11 20:01:56.503 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:makeReport
Apr-11 20:01:56.503 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > output
Apr-11 20:01:56.504 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
Apr-11 20:01:56.504 [main] DEBUG nextflow.Session - Session await
Apr-11 20:01:56.797 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:01:56.804 [Task submitter] INFO  nextflow.Session - [ac/9b3ef4] Submitted process > fastcat (1)
Apr-11 20:02:44.286 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: fastcat (1); status: COMPLETED; exit: 0; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ac/9b3ef4b292e6f0e51d04cadf5720e8]
Apr-11 20:02:44.292 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:02:44.295 [Task submitter] INFO  nextflow.Session - [56/ab90db] Submitted process > pipeline:getVersions
Apr-11 20:02:44.301 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:02:44.302 [Task submitter] INFO  nextflow.Session - [dc/654298] Submitted process > pipeline:getParams
Apr-11 20:02:44.400 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:02:44.401 [Task submitter] INFO  nextflow.Session - [4f/2432f9] Submitted process > pipeline:collectFastqIngressResultsInDir (1)
Apr-11 20:02:47.251 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: pipeline:collectFastqIngressResultsInDir (1); status: COMPLETED; exit: 0; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/4f/2432f9bf0abed39e289644823dfbd5]
Apr-11 20:02:47.371 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: pipeline:getParams; status: COMPLETED; exit: 0; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/dc/654298d659f58ed76dae351a33e0d4]
Apr-11 20:02:47.381 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:02:47.410 [Task submitter] INFO  nextflow.Session - [e3/c29097] Submitted process > output (1)
Apr-11 20:02:49.154 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 6; name: output (1); status: COMPLETED; exit: 0; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/e3/c29097b30caedaa82bba8fe22944d1]
Apr-11 20:02:49.169 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'PublishDir' minSize=10; maxSize=624; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Apr-11 20:02:53.736 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: pipeline:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/56/ab90db8faec36e76898a82a716ea86]
Apr-11 20:02:53.746 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:02:53.747 [Task submitter] INFO  nextflow.Session - [ed/5293ac] Submitted process > pipeline:preprocess_reads (1)
Apr-11 20:06:56.174 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:11:56.192 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:16:56.290 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:21:56.326 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:26:56.374 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:31:56.466 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:36:56.485 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:41:56.521 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:46:56.576 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:51:56.649 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: RUNNING; exit: -; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:54:02.167 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 5; name: pipeline:preprocess_reads (1); status: COMPLETED; exit: 0; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ed/5293ac30bf62fff67533ade8521b9f]
Apr-11 20:54:02.259 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:54:02.260 [Task submitter] INFO  nextflow.Session - [90/f1b4f3] Submitted process > pipeline:denovo_assembly:make_batches (1)
Apr-11 20:54:02.298 [Actor Thread 127] DEBUG nextflow.sort.BigSort - Sort completed -- entries: 1; slices: 1; internal sort time: 0.013 s; external sort time: 0.044 s; total time: 0.057 s
Apr-11 20:54:02.664 [Actor Thread 127] DEBUG nextflow.file.FileCollector - Saved collect-files list to: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/collect-file/cd9f347dd5457c48f51f7222509b452f
Apr-11 20:54:02.680 [Actor Thread 127] DEBUG nextflow.file.FileCollector - Deleting file collector temp dir: /tmp/nxf-12462363521678353820
Apr-11 20:54:46.248 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 7; name: pipeline:denovo_assembly:make_batches (1); status: COMPLETED; exit: 0; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/90/f1b4f3550d7e23a3bcbaeddf03d849]
Apr-11 20:54:46.321 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-11 20:54:46.322 [Task submitter] INFO  nextflow.Session - [ab/c4ddcb] Submitted process > pipeline:denovo_assembly:clustering (1)
Apr-11 20:54:55.950 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 8; name: pipeline:denovo_assembly:clustering (1); status: COMPLETED; exit: 2; error: -; workDir: /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ab/c4ddcbe7604006308a3b620f2c67c1]
Apr-11 20:54:55.957 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=pipeline:denovo_assembly:clustering (1); work-dir=/storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ab/c4ddcbe7604006308a3b620f2c67c1
  error [nextflow.exception.ProcessFailedException]: Process `pipeline:denovo_assembly:clustering (1)` terminated with an error exit status (2)
Apr-11 20:54:56.013 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'pipeline:denovo_assembly:clustering (1)'

Caused by:
  Process `pipeline:denovo_assembly:clustering (1)` terminated with an error exit status (2)

Command executed:

  workflow-glue run_isonclust2 batches

Command exit status:
  2

Command output:
  (empty)

Command error:
  /home/epi2melabs/conda/lib/python3.8/site-packages/gffutils/parser.py:19: DeprecationWarning: invalid escape sequence \w
    gff3_kw_pat = re.compile('\w+=')
  usage: wf-glue [-h] [--debug | --quiet] [-v]
                 {check_sample_sheet,compute_cluster_quality,generate_pychopper_stats,generate_tracking_summary,merge_count_tsvs,report,run_isonclust2}
                 ...
  wf-glue: error: unrecognized arguments: batches

Work dir:
  /storage-02/zhaoqiangsen/female_transcripts_RNAseq/chicken_asm/ONT/female_workspace_dir/ab/c4ddcbe7604006308a3b620f2c67c1

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
Apr-11 20:54:56.025 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `pipeline:denovo_assembly:clustering (1)` terminated with an error exit status (2)
Apr-11 20:54:56.093 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
[process] pipeline:denovo_assembly:merge_cds

[process] pipeline:denovo_assembly:cds_align
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

[process] pipeline:split_bam
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

[process] pipeline:assemble_transcripts
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: ref_annotation
  port 2: (cntrl) -     ; channel: $

[process] pipeline:merge_gff_bundles
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

[process] pipeline:run_gffcompare
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: ref_annotation
  port 2: (cntrl) -     ; channel: $

[process] pipeline:get_transcriptome
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

[process] pipeline:makeReport
  status=ACTIVE
  port 0: (value) bound ; channel: versions
  port 1: (value) bound ; channel: params.json
  port 2: (queue) OPEN  ; channel: pychopper_report/*
  port 3: (value) bound ; channel: jaffal_csv/*
  port 4: (value) bound ; channel: sample_ids
  port 5: (queue) OPEN  ; channel: per_read_stats
  port 6: (value) OPEN  ; channel: aln_stats/*
  port 7: (value) OPEN  ; channel: gffcmp_dir
  port 8: (value) OPEN  ; channel: gff_annotation/*
  port 9: (value) bound ; channel: de_report/*
  port 10: (value) bound ; channel: seqkit/*
  port 11: (cntrl) -     ; channel: $

[process] output
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

Apr-11 20:54:56.367 [main] DEBUG nextflow.Session - Session await > all processes finished
Apr-11 20:54:56.997 [main] DEBUG nextflow.Session - Session await > all barriers passed
Apr-11 20:54:57.296 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=7; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=3h 28m 9s; failedDuration=9.6s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=3; peakCpus=4; peakMemory=0; ]
Apr-11 20:54:57.296 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Apr-11 20:54:57.300 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Apr-11 20:54:57.506 [main] DEBUG nextflow.trace.ReportObserver - Execution report summary data:
  [{"cpuUsage":{"mean":63.85,"min":63.85,"q1":63.85,"q2":63.85,"q3":63.85,"max":63.85,"minLabel":"fastcat (1)","maxLabel":"fastcat (1)","q1Label":"fastcat (1)","q2Label":"fastcat (1)","q3Label":"fastcat (1)"},"process":"fastcat","mem":{"mean":7413760,"min":7413760,"q1":7413760,"q2":7413760,"q3":7413760,"max":7413760,"minLabel":"fastcat (1)","maxLabel":"fastcat (1)","q1Label":"fastcat (1)","q2Label":"fastcat (1)","q3Label":"fastcat (1)"},"memUsage":null,"timeUsage":null,"vmem":{"mean":395440128,"min":395440128,"q1":395440128,"q2":395440128,"q3":395440128,"max":395440128,"minLabel":"fastcat (1)","maxLabel":"fastcat (1)","q1Label":"fastcat (1)","q2Label":"fastcat (1)","q3Label":"fastcat (1)"},"reads":{"mean":4235578836,"min":4235578836,"q1":4235578836,"q2":4235578836,"q3":4235578836,"max":4235578836,"minLabel":"fastcat (1)","maxLabel":"fastcat (1)","q1Label":"fastcat (1)","q2Label":"fastcat (1)","q3Label":"fastcat (1)"},"cpu":{"mean":255.4,"min":255.4,"q1":255.4,"q2":255.4,"q3":255.4,"max":255.4,"minLabel":"fastcat (1)","maxLabel":"fastcat (1)","q1Label":"fastcat (1)","q2Label":"fastcat (1)","q3Label":"fastcat (1)"},"time":{"mean":45105,"min":45105,"q1":45105,"q2":45105,"q3":45105,"max":45105,"minLabel":"fastcat (1)","maxLabel":"fastcat (1)","q1Label":"fastcat (1)","q2Label":"fastcat (1)","q3Label":"fastcat (1)"},"writes":{"mean":4817445307,"min":4817445307,"q1":4817445307,"q2":4817445307,"q3":4817445307,"max":4817445307,"minLabel":"fastcat (1)","maxLabel":"fastcat (1)","q1Label":"fastcat (1)","q2Label":"fastcat (1)","q3Label":"fastcat (1)"}},{"cpuUsage":{"mean":52,"min":52,"q1":52,"q2":52,"q3":52,"max":52,"minLabel":"pipeline:collectFastqIngressResultsInDir (1)","maxLabel":"pipeline:collectFastqIngressResultsInDir (1)","q1Label":"pipeline:collectFastqIngressResultsInDir (1)","q2Label":"pipeline:collectFastqIngressResultsInDir (1)","q3Label":"pipeline:collectFastqIngressResultsInDir (1)"},"process":"collectFastqIngressResultsInDir","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":{"mean":95965,"min":95965,"q1":95965,"q2":95965,"q3":95965,"max":95965,"minLabel":"pipeline:collectFastqIngressResultsInDir (1)","maxLabel":"pipeline:collectFastqIngressResultsInDir (1)","q1Label":"pipeline:collectFastqIngressResultsInDir (1)","q2Label":"pipeline:collectFastqIngressResultsInDir (1)","q3Label":"pipeline:collectFastqIngressResultsInDir (1)"},"cpu":{"mean":52,"min":52,"q1":52,"q2":52,"q3":52,"max":52,"minLabel":"pipeline:collectFastqIngressResultsInDir (1)","maxLabel":"pipeline:collectFastqIngressResultsInDir (1)","q1Label":"pipeline:collectFastqIngressResultsInDir (1)","q2Label":"pipeline:collectFastqIngressResultsInDir (1)","q3Label":"pipeline:collectFastqIngressResultsInDir (1)"},"time":{"mean":10,"min":10,"q1":10,"q2":10,"q3":10,"max":10,"minLabel":"pipeline:collectFastqIngressResultsInDir (1)","maxLabel":"pipeline:collectFastqIngressResultsInDir (1)","q1Label":"pipeline:collectFastqIngressResultsInDir (1)","q2Label":"pipeline:collectFastqIngressResultsInDir (1)","q3Label":"pipeline:collectFastqIngressResultsInDir (1)"},"writes":{"mean":289,"min":289,"q1":289,"q2":289,"q3":289,"max":289,"minLabel":"pipeline:collectFastqIngressResultsInDir (1)","maxLabel":"pipeline:collectFastqIngressResultsInDir (1)","q1Label":"pipeline:collectFastqIngressResultsInDir (1)","q2Label":"pipeline:collectFastqIngressResultsInDir (1)","q3Label":"pipeline:collectFastqIngressResultsInDir (1)"}},{"cpuUsage":{"mean":107,"min":107,"q1":107,"q2":107,"q3":107,"max":107,"minLabel":"pipeline:getParams","maxLabel":"pipeline:getParams","q1Label":"pipeline:getParams","q2Label":"pipeline:getParams","q3Label":"pipeline:getParams"},"process":"getParams","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":{"mean":80207,"min":80207,"q1":80207,"q2":80207,"q3":80207,"max":80207,"minLabel":"pipeline:getParams","maxLabel":"pipeline:getParams","q1Label":"pipeline:getParams","q2Label":"pipeline:getParams","q3Label":"pipeline:getParams"},"cpu":{"mean":107,"min":107,"q1":107,"q2":107,"q3":107,"max":107,"minLabel":"pipeline:getParams","maxLabel":"pipeline:getParams","q1Label":"pipeline:getParams","q2Label":"pipeline:getParams","q3Label":"pipeline:getParams"},"time":{"mean":11,"min":11,"q1":11,"q2":11,"q3":11,"max":11,"minLabel":"pipeline:getParams","maxLabel":"pipeline:getParams","q1Label":"pipeline:getParams","q2Label":"pipeline:getParams","q3Label":"pipeline:getParams"},"writes":{"mean":2591,"min":2591,"q1":2591,"q2":2591,"q3":2591,"max":2591,"minLabel":"pipeline:getParams","maxLabel":"pipeline:getParams","q1Label":"pipeline:getParams","q2Label":"pipeline:getParams","q3Label":"pipeline:getParams"}},{"cpuUsage":{"mean":78.9,"min":78.9,"q1":78.9,"q2":78.9,"q3":78.9,"max":78.9,"minLabel":"output (1)","maxLabel":"output (1)","q1Label":"output (1)","q2Label":"output (1)","q3Label":"output (1)"},"process":"output","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":{"mean":77714,"min":77714,"q1":77714,"q2":77714,"q3":77714,"max":77714,"minLabel":"output (1)","maxLabel":"output (1)","q1Label":"output (1)","q2Label":"output (1)","q3Label":"output (1)"},"cpu":{"mean":78.9,"min":78.9,"q1":78.9,"q2":78.9,"q3":78.9,"max":78.9,"minLabel":"output (1)","maxLabel":"output (1)","q1Label":"output (1)","q2Label":"output (1)","q3Label":"output (1)"},"time":{"mean":5,"min":5,"q1":5,"q2":5,"q3":5,"max":5,"minLabel":"output (1)","maxLabel":"output (1)","q1Label":"output (1)","q2Label":"output (1)","q3Label":"output (1)"},"writes":{"mean":215,"min":215,"q1":215,"q2":215,"q3":215,"max":215,"minLabel":"output (1)","maxLabel":"output (1)","q1Label":"output (1)","q2Label":"output (1)","q3Label":"output (1)"}},{"cpuUsage":{"mean":368.6,"min":368.6,"q1":368.6,"q2":368.6,"q3":368.6,"max":368.6,"minLabel":"pipeline:getVersions","maxLabel":"pipeline:getVersions","q1Label":"pipeline:getVersions","q2Label":"pipeline:getVersions","q3Label":"pipeline:getVersions"},"process":"getVersions","mem":{"mean":73707520,"min":73707520,"q1":73707520,"q2":73707520,"q3":73707520,"max":73707520,"minLabel":"pipeline:getVersions","maxLabel":"pipeline:getVersions","q1Label":"pipeline:getVersions","q2Label":"pipeline:getVersions","q3Label":"pipeline:getVersions"},"memUsage":null,"timeUsage":null,"vmem":{"mean":18250661888,"min":18250661888,"q1":18250661888,"q2":18250661888,"q3":18250661888,"max":18250661888,"minLabel":"pipeline:getVersions","maxLabel":"pipeline:getVersions","q1Label":"pipeline:getVersions","q2Label":"pipeline:getVersions","q3Label":"pipeline:getVersions"},"reads":{"mean":38430119,"min":38430119,"q1":38430119,"q2":38430119,"q3":38430119,"max":38430119,"minLabel":"pipeline:getVersions","maxLabel":"pipeline:getVersions","q1Label":"pipeline:getVersions","q2Label":"pipeline:getVersions","q3Label":"pipeline:getVersions"},"cpu":{"mean":368.6,"min":368.6,"q1":368.6,"q2":368.6,"q3":368.6,"max":368.6,"minLabel":"pipeline:getVersions","maxLabel":"pipeline:getVersions","q1Label":"pipeline:getVersions","q2Label":"pipeline:getVersions","q3Label":"pipeline:getVersions"},"time":{"mean":5862,"min":5862,"q1":5862,"q2":5862,"q3":5862,"max":5862,"minLabel":"pipeline:getVersions","maxLabel":"pipeline:getVersions","q1Label":"pipeline:getVersions","q2Label":"pipeline:getVersions","q3Label":"pipeline:getVersions"},"writes":{"mean":19192591,"min":19192591,"q1":19192591,"q2":19192591,"q3":19192591,"max":19192591,"minLabel":"pipeline:getVersions","maxLabel":"pipeline:getVersions","q1Label":"pipeline:getVersions","q2Label":"pipeline:getVersions","q3Label":"pipeline:getVersions"}},{"cpuUsage":{"mean":73.35,"min":73.35,"q1":73.35,"q2":73.35,"q3":73.35,"max":73.35,"minLabel":"pipeline:preprocess_reads (1)","maxLabel":"pipeline:preprocess_reads (1)","q1Label":"pipeline:preprocess_reads (1)","q2Label":"pipeline:preprocess_reads (1)","q3Label":"pipeline:preprocess_reads (1)"},"process":"preprocess_reads","mem":{"mean":7012032512,"min":7012032512,"q1":7012032512,"q2":7012032512,"q3":7012032512,"max":7012032512,"minLabel":"pipeline:preprocess_reads (1)","maxLabel":"pipeline:preprocess_reads (1)","q1Label":"pipeline:preprocess_reads (1)","q2Label":"pipeline:preprocess_reads (1)","q3Label":"pipeline:preprocess_reads (1)"},"memUsage":null,"timeUsage":null,"vmem":{"mean":94006718464,"min":94006718464,"q1":94006718464,"q2":94006718464,"q3":94006718464,"max":94006718464,"minLabel":"pipeline:preprocess_reads (1)","maxLabel":"pipeline:preprocess_reads (1)","q1Label":"pipeline:preprocess_reads (1)","q2Label":"pipeline:preprocess_reads (1)","q3Label":"pipeline:preprocess_reads (1)"},"reads":{"mean":15423943246,"min":15423943246,"q1":15423943246,"q2":15423943246,"q3":15423943246,"max":15423943246,"minLabel":"pipeline:preprocess_reads (1)","maxLabel":"pipeline:preprocess_reads (1)","q1Label":"pipeline:preprocess_reads (1)","q2Label":"pipeline:preprocess_reads (1)","q3Label":"pipeline:preprocess_reads (1)"},"cpu":{"mean":293.4,"min":293.4,"q1":293.4,"q2":293.4,"q3":293.4,"max":293.4,"minLabel":"pipeline:preprocess_reads (1)","maxLabel":"pipeline:preprocess_reads (1)","q1Label":"pipeline:preprocess_reads (1)","q2Label":"pipeline:preprocess_reads (1)","q3Label":"pipeline:preprocess_reads (1)"},"time":{"mean":3065640,"min":3065640,"q1":3065640,"q2":3065640,"q3":3065640,"max":3065640,"minLabel":"pipeline:preprocess_reads (1)","maxLabel":"pipeline:preprocess_reads (1)","q1Label":"pipeline:preprocess_reads (1)","q2Label":"pipeline:preprocess_reads (1)","q3Label":"pipeline:preprocess_reads (1)"},"writes":{"mean":11601439754,"min":11601439754,"q1":11601439754,"q2":11601439754,"q3":11601439754,"max":11601439754,"minLabel":"pipeline:preprocess_reads (1)","maxLabel":"pipeline:preprocess_reads (1)","q1Label":"pipeline:preprocess_reads (1)","q2Label":"pipeline:preprocess_reads (1)","q3Label":"pipeline:preprocess_reads (1)"}},{"cpuUsage":{"mean":213.6,"min":213.6,"q1":213.6,"q2":213.6,"q3":213.6,"max":213.6,"minLabel":"pipeline:denovo_assembly:make_batches (1)","maxLabel":"pipeline:denovo_assembly:make_batches (1)","q1Label":"pipeline:denovo_assembly:make_batches (1)","q2Label":"pipeline:denovo_assembly:make_batches (1)","q3Label":"pipeline:denovo_assembly:make_batches (1)"},"process":"make_batches","mem":{"mean":1852411904,"min":1852411904,"q1":1852411904,"q2":1852411904,"q3":1852411904,"max":1852411904,"minLabel":"pipeline:denovo_assembly:make_batches (1)","maxLabel":"pipeline:denovo_assembly:make_batches (1)","q1Label":"pipeline:denovo_assembly:make_batches (1)","q2Label":"pipeline:denovo_assembly:make_batches (1)","q3Label":"pipeline:denovo_assembly:make_batches (1)"},"memUsage":null,"timeUsage":null,"vmem":{"mean":14389837824,"min":14389837824,"q1":14389837824,"q2":14389837824,"q3":14389837824,"max":14389837824,"minLabel":"pipeline:denovo_assembly:make_batches (1)","maxLabel":"pipeline:denovo_assembly:make_batches (1)","q1Label":"pipeline:denovo_assembly:make_batches (1)","q2Label":"pipeline:denovo_assembly:make_batches (1)","q3Label":"pipeline:denovo_assembly:make_batches (1)"},"reads":{"mean":396186288,"min":396186288,"q1":396186288,"q2":396186288,"q3":396186288,"max":396186288,"minLabel":"pipeline:denovo_assembly:make_batches (1)","maxLabel":"pipeline:denovo_assembly:make_batches (1)","q1Label":"pipeline:denovo_assembly:make_batches (1)","q2Label":"pipeline:denovo_assembly:make_batches (1)","q3Label":"pipeline:denovo_assembly:make_batches (1)"},"cpu":{"mean":213.6,"min":213.6,"q1":213.6,"q2":213.6,"q3":213.6,"max":213.6,"minLabel":"pipeline:denovo_assembly:make_batches (1)","maxLabel":"pipeline:denovo_assembly:make_batches (1)","q1Label":"pipeline:denovo_assembly:make_batches (1)","q2Label":"pipeline:denovo_assembly:make_batches (1)","q3Label":"pipeline:denovo_assembly:make_batches (1)"},"time":{"mean":40200,"min":40200,"q1":40200,"q2":40200,"q3":40200,"max":40200,"minLabel":"pipeline:denovo_assembly:make_batches (1)","maxLabel":"pipeline:denovo_assembly:make_batches (1)","q1Label":"pipeline:denovo_assembly:make_batches (1)","q2Label":"pipeline:denovo_assembly:make_batches (1)","q3Label":"pipeline:denovo_assembly:make_batches (1)"},"writes":{"mean":1038985676,"min":1038985676,"q1":1038985676,"q2":1038985676,"q3":1038985676,"max":1038985676,"minLabel":"pipeline:denovo_assembly:make_batches (1)","maxLabel":"pipeline:denovo_assembly:make_batches (1)","q1Label":"pipeline:denovo_assembly:make_batches (1)","q2Label":"pipeline:denovo_assembly:make_batches (1)","q3Label":"pipeline:denovo_assembly:make_batches (1)"}},{"cpuUsage":null,"process":"clustering","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":null,"cpu":null,"time":{"mean":9601,"min":9601,"q1":9601,"q2":9601,"q3":9601,"max":9601,"minLabel":"pipeline:denovo_assembly:clustering (1)","maxLabel":"pipeline:denovo_assembly:clustering (1)","q1Label":"pipeline:denovo_assembly:clustering (1)","q2Label":"pipeline:denovo_assembly:clustering (1)","q3Label":"pipeline:denovo_assembly:clustering (1)"},"writes":null}]
Apr-11 20:54:58.349 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Apr-11 20:54:58.484 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Apr-11 20:54:58.518 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
CWYuan08 commented 1 year ago

Dear @Johnsonzcode,

many thanks for your comments under my issue, I am wondering what's the command you used here?

Best, CW

Caffeinated-Code commented 1 year ago

Getting the same error while running the nextflow pipeline at the CLI on AWS cloud9 environment (Amazon Linux 2) Looks like a python issue of escaping backslashes.

Caused by: Process pipeline:denovo_assembly:clustering (1) terminated with an error exit status (2)

Command executed:

workflow-glue run_isonclust2 batches

Command exit status: 2

Command output: (empty)

Command error: /home/epi2melabs/conda/lib/python3.8/site-packages/gffutils/parser.py:19: DeprecationWarning: invalid escape sequence \w gff3_kw_pat = re.compile('\w+=') usage: wf-glue [-h] [--debug | --quiet] [-v] {check_sample_sheet,compute_cluster_quality,generate_pychopper_stats,generate_tracking_summary,merge_count_tsvs,report,run_isonclust2} ... wf-glue: error: unrecognized arguments: batches

cjw85 commented 1 year ago

Thank you all for reporting this error. We'll pick it up as soon as we can. I'd like to first understand why this wasn't picked up in our automated testing 🤔

sarahjeeeze commented 1 year ago

Hi, This should be fixed in the latest version v0.1.13. Thanks again for reporting.

CWYuan08 commented 1 year ago

Dear @sarahjeeeze,

many thanks for the update, I have tried to run v0.1.13, I am getting a new error: it stuck at the make_batches stage, Error executing process > 'output (1)'

Caused by: Cannot run program "/bin/bash" (in directory "PromethION_RNA/epi2me_denovo/w orkspace/e7/0a59ec4cbf2ced583fbdb25723030d"): error=2, No such file or directory I checked the directory does exist, and the slurm error is: [WARN] Task failed java.io.IOException: Cannot run program "sh": error=2, No such file or directory at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143) at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073) at java.base/java.lang.Runtime.exec(Runtime.java:594) at java.base/java.lang.Runtime.exec(Runtime.java:453) at jline.internal.TerminalLineSettings.exec(TerminalLineSettings.java:183) at jline.internal.TerminalLineSettings.exec(TerminalLineSettings.java:173) at jline.internal.TerminalLineSettings.stty(TerminalLineSettings.java:168) at jline.internal.TerminalLineSettings.set(TerminalLineSettings.java:76) at jline.internal.TerminalLineSettings.restore(TerminalLineSettings.java:68) at jline.UnixTerminal.restore(UnixTerminal.java:65) at jline.TerminalSupport$1.run(TerminalSupport.java:49) at jline.internal.ShutdownHooks.runTasks(ShutdownHooks.java:66) at jline.internal.ShutdownHooks.access$000(ShutdownHooks.java:22) at jline.internal.ShutdownHooks$1.run(ShutdownHooks.java:47) Caused by: java.io.IOException: error=2, No such file or directory at java.base/java.lang.ProcessImpl.forkAndExec(Native Method) at java.base/java.lang.ProcessImpl.(ProcessImpl.java:314) at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244) at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110) ... 13 more

Could you please help me on this?

Thank you very much!

Best, CW

Caffeinated-Code commented 1 year ago

Hi @sarahjeeeze I am replicating the same "make batches" error as @CWYuan08 on an internal dataset processing it on AWS Cloud9 (Amazon Linux 2), profile=docker.

Nextflow log Error executing process > 'pipeline:denovo_assembly:clustering (1)'

Caused by: Process pipeline:denovo_assembly:clustering (1) terminated with an error exit status (2)

Command executed:

workflow-glue run_isonclust2 batches

Command exit status: 2

Command output: (empty)

Command error: /home/epi2melabs/conda/lib/python3.8/site-packages/gffutils/parser.py:19: DeprecationWarning: invalid escape sequence \w gff3_kw_pat = re.compile('\w+=') usage: wf-glue [-h] [--debug | --quiet] [-v] {check_sample_sheet,compute_cluster_quality,generate_pychopper_stats,generate_tracking_summary,merge_count_tsvs,report,run_isonclust2} ... wf-glue: error: unrecognized arguments: batches

Command run

!/bin/bash -euo pipefail

workflow-glue run_isonclust2 batches

Command err /home/epi2melabs/conda/lib/python3.8/site-packages/gffutils/parser.py:19: DeprecationWarning: invalid escape sequence \w gff3_kw_pat = re.compile('\w+=') usage: wf-glue [-h] [--debug | --quiet] [-v] {check_sample_sheet,compute_cluster_quality,generate_pychopper_stats,generate_tracking_summary,merge_count_tsvs,report,run_isonclust2} ... wf-glue: error: unrecognized arguments: batches

ETA: The pipeline runs for close to ~2-3hrs at the make batches step prior to crashing

sarahjeeeze commented 1 year ago

Hi @Caffeinated-Code which version of the workflow are you using? v0.1.13? - that error looks like one from an older version.

sarahjeeeze commented 1 year ago

Hi @CWYuan08, I am not sure exactly what is causing that java error but looks like its related to your environment, if you are running it on a device does it have enough space on it? If not perhaps set up a mount point for the work directory if possible, and clean up any unused work directories with nextflow clean

Caffeinated-Code commented 1 year ago

Hi Sarah, I have revised it to the recent v0.1.13 and getting another workflow glue error.

Core Nextflow options revision : v0.1.13 runName : cranky_lumiere containerEngine : docker launchDir : /home/ec2-user/environment/scripts workDir : /data/workdir projectDir : /home/ec2-user/.nextflow/assets/epi2me-labs/wf-transcriptomes userName : ec2-user profile : standard configFiles : /home/ec2-user/.nextflow/config, /home/ec2-user/.nextflow/assets/epi2me-labs/wf-transcriptomes/nextflow.config

Input Options fastq : /data/Downstream_Analaysis/xxx/RawData transcriptome_source : denovo ref_genome : /home/ec2-user/environment/annotations/minimap2/hg38as.fa ref_transcriptome : null

Output Options out_dir : /data/TEST_wf_transcriptomes.DeNovo.xxx.202304

Options for reference-based workflow plot_gffcmp_stats : true minimap2_opts : -y -p 0.99 -k14 --MD -ax splice

Options for de novo-based workflow isOnClust2_sort_options: --batch-size -1 --kmer-size 11 --window-size 15 --min-shared 5 --min-qual 7.0 --mapped-threshold 0.65 --aligned-threshold 0.2 --min-fraction 0.8 --min-prob-no-hits 0.0 -M -1 -P 500 -g 50 -c 150 -F 2

Differential Expression Options condition_sheet : test_data/condition_sheet.tsv

Advanced Options threads : 4 pychopper_opts : -m phmm -k PCS111 -U -y bundle_min_reads : 50000

executor > local (1362) [97/f5cb92] process > fastcat (1) [100%] 1 of 1 ✔ [c4/9f9a16] process > pipeline:preprocess_ref_annotation [100%] 1 of 1 ✔ [95/50cd41] process > pipeline:collectFastqIngressResultsInDir (1) [100%] 1 of 1 ✔ [53/d79776] process > pipeline:getVersions [100%] 1 of 1 ✔ [64/3cd7ad] process > pipeline:getParams [100%] 1 of 1 ✔ [5f/0734f3] process > pipeline:preprocess_reads (1) [100%] 1 of 1 ✔ [c3/1dfad6] process > pipeline:denovo_assembly:make_batches (1) [100%] 1 of 1 ✔ [90/bb8e68] process > pipeline:denovo_assembly:clustering (1) [100%] 1 of 1 ✔ [54/fcf0e8] process > pipeline:denovo_assembly:dump_clusters (1) [100%] 1 of 1 ✔ [b0/b427e4] process > pipeline:denovo_assembly:build_backbones (1256) [ 3%] 1347 of 41501 [- ] process > pipeline:denovo_assembly:merge_cds - [- ] process > pipeline:denovo_assembly:cds_align - [7e/b0bf1c] process > pipeline:denovo_assembly:cluster_quality (1) [ 0%] 0 of 1 [- ] process > pipeline:split_bam - [- ] process > pipeline:assemble_transcripts - [- ] process > pipeline:merge_gff_bundles - [- ] process > pipeline:run_gffcompare - [- ] process > pipeline:get_transcriptome - [- ] process > pipeline:makeReport - [bd/77c0b3] process > output (1) [100%] 1 of 1 Error executing process > 'pipeline:denovo_assembly:cluster_quality (1)'

Caused by: Process pipeline:denovo_assembly:cluster_quality (1) terminated with an error exit status (1)

Command executed:

mkdir RawData_cluster_qc mkdir RawData_cluster_qc_raw minimap2 -ax splice -t 2 hg38as.fa RawData_full_length_reads.fastq | samtools view -q 2 -F 2304 -b - | samtools sort - -o RawData_cluster_qc/ref_aln.bam; samtools index RawData_cluster_qc/ref_aln.bam; workflow-glue compute_cluster_quality --sizes final_clusters/clusters_info.tsv --outfile RawData_cluster_qc/cluster_quality.csv --ont --clusters final_clusters/clusters.tsv --classes RawData_cluster_qc/ref_aln.bam --report RawData_cluster_qc/cluster_quality.pdf --raw_data_out RawData_cluster_qc_raw

Command exit status: 1

Command output: (empty)

Command error: [M::mm_idx_gen::133.8070.97] collected minimizers [M::mm_idx_gen::209.9550.97] sorted minimizers [M::main::209.9560.97] loaded/built the index for 2677 target sequence(s) [M::mm_mapopt_update::213.7060.97] mid_occ = 753 [M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 2677 [M::mm_idx_stat::215.9020.97] distinct minimizers: 167346621 (35.44% are singletons); average occurrences: 6.015; average spacing: 3.085; total length: 3105806778 [M::worker_pipeline::1508.4650.98] mapped 4376076 sequences [M::worker_pipeline::2188.3510.98] mapped 4046703 sequences [M::worker_pipeline::2209.6840.98] mapped 181916 sequences [M::main] Version: 2.24-r1122 [M::main] CMD: minimap2 -ax splice -t 2 hg38as.fa RawData_full_length_reads.fastq [M::main] Real time: 2211.288 sec; CPU: 2173.047 sec; Peak RSS: 18.386 GB /home/epi2melabs/conda/lib/python3.8/site-packages/gffutils/parser.py:19: DeprecationWarning: invalid escape sequence \w gff3_kw_pat = re.compile('\w+=') [01:08:08 - workflow_glue] Starting entrypoint. Traceback (most recent call last): File "/home/ec2-user/.nextflow/assets/epi2me-labs/wf-transcriptomes/bin/workflow-glue", line 7, in cli() File "/home/ec2-user/.nextflow/assets/epi2me-labs/wf-transcriptomes/bin/workflow_glue/init.py", line 62, in cli args.func(args) File "/home/ec2-user/.nextflow/assets/epi2me-labs/wf-transcriptomes/bin/workflow_glue/compute_cluster_quality.py", line 439, in main clusters = parse_inferred_clusters_tsv(args.clusters, args) File "/home/ec2-user/.nextflow/assets/epi2me-labs/wf-transcriptomes/bin/workflow_glue/compute_cluster_quality.py", line 77, in parse_inferred_clusters_tsv clusterid, , read_acc = line.strip().split("\t") ValueError: too many values to unpack (expected 3)

sarahjeeeze commented 1 year ago

Hi, thanks for the feedback, i'll try to recreate this error and get back to you.

CWYuan08 commented 1 year ago

Dear @sarahjeeeze, I am stuck at the exact same step as @Caffeinated-Code at pipeline:denovo_assembly:build_backbones, any help on this will be greatly appreciated!! Best, CW

Caffeinated-Code commented 1 year ago

Hi @sarahjeeeze, Were you able to recreate the most recent error on your end? @CWYuan08

sarahjeeeze commented 1 year ago

Hi, I am able to recreate the cluster_quality error if there are no alignments between the clusters and the supplied ref_genome. I will try to improve the error messages, if you run the denovo workflow without the ref_genome it should at least complete, but you will be missing the cluster quality data.. maybe insure the ref_genome you are using is relevant to the input data.

CWYuan08 commented 1 year ago

Dear @sarahjeeeze,

thank you for clarifying this, I wish to use our reference for this analysis (it is being used in other analysis without problem), is there anyway to separate out the clusters if there are no alignments? I am rerunning the pipeline now.

Btw, I am also wondering if there is an upper limit for the data size, I stuck at cluster_quality when using a MinION data (~6M reads), but when I try it with our PromethION data, it stuck again at making batches. Many thanks again!

Best, CW

sarahjeeeze commented 1 year ago

Hi, Sorry for the late response. We would have to add something to the pipeline to seperate out the clusters if there are no alignments. There is not an upper limit for the data size but if you are using large samples you may want to adjust the computational resources available to the workflow and processes by adjusting the config. What is the error you get from the making batches process?

sarahjeeeze commented 1 year ago

Closing through lack of response.