search

epi2me-labs / wf-single-cell

Other
75 stars 39 forks source link

RuntimeError: cannot cache function 'rdist': no locator available #121

Closed sme229 closed 4 months ago

sme229 commented 4 months ago

Operating System

Other Linux (please specify below)

Other Linux

Linux 5.14.21-150400.24.28-default

Workflow Version

wf-single-cell v2.1.0-ga25ec6c

Workflow Execution

Command line (Cluster)

Other workflow execution

No response

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-single-cell --expected_cells 3000 -profile singularity --fastq 'Elena-cDNA-2/20240624_1312_1D_PAW58377_76308a2c/fastq_pass' --kit_name '3prime' --kit_version 'v3' --ref_genome_dir 'OryCun_genome'

Workflow Execution - CLI Execution Profile

singularity

What happened?

Hello, could you please help with Numba issue. First, I had a different Numba-related error and updating Numba helped to resolve that issue. Now, after ~9 h of running the workflow, I get this error: File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 337, in init raise RuntimeError("cannot cache function %r: no locator available " RuntimeError: cannot cache function 'rdist': no locator available for file '/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py'

Relevant log output

pipeline epi2me
Traceback (most recent call last):
  File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in <module>
    cli()
  File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 66, in cli
    components = get_components(allowed_components=[sys.argv[1]])
  File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 29, in get_components
    mod = importlib.import_module(f"{_package_name}.{name}")
  File "/home/epi2melabs/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/process_matrix.py", line 8, in <module>
    import umap
  File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/__init__.py", line 2, in <module>
    from .umap_ import UMAP
  File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py", line 41, in <module>
    from umap.layouts import (
  File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py", line 40, in <module>
    def rdist(x, y):
  File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/decorators.py", line 234, in wrapper
    disp.enable_caching()
  File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/dispatcher.py", line 863, in enable_caching
    self._cache = FunctionCache(self.py_func)
  File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 601, in init
    self._impl = self._impl_class(py_func)
  File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 337, in init
    raise RuntimeError("cannot cache function %r: no locator available "
RuntimeError: cannot cache function 'rdist': no locator available for file '/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py'

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

no

Other demo data information

I ran into this error:

Jul-17 17:43:38.875 [main] DEBUG nextflow.cli.Launcher - $> nextflow run epi2me-labs/wf-single-cell --expected_cells 100 -profile singularity --fastq wf-single-cell-demo/chr17.fq.gz --kit_name 3prime --kit_version v3 --ref_genome_dir wf-single-cell-demo --umap_plot_genes wf-single-cell-demo/umap_plot_genes.csv
Jul-17 17:43:39.020 [main] DEBUG nextflow.cli.CmdRun - N E X T F L O W  ~  version 24.04.2
Jul-17 17:43:39.036 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/sme229/.nextflow/plugins; core-plugins: nf-amazon@2.5.2,nf-azure@1.6.0,nf-cloudcache@0.4.1,nf-codecommit@0.2.0,nf-console@1.1.3,nf-ga4gh@1.3.0,nf-google@1.13.2,nf-tower@1.9.1,nf-wave@1.4.2
Jul-17 17:43:39.045 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-17 17:43:39.046 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-17 17:43:39.048 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.10.0 in 'deployment' mode
Jul-17 17:43:39.058 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jul-17 17:43:39.067 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/sme229/.nextflow/scm
Jul-17 17:43:39.606 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-single-cell.git
Jul-17 17:43:39.627 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Jul-17 17:43:39.634 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-single-cell.git
Jul-17 17:43:40.528 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/nextflow.config
Jul-17 17:43:40.529 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/nextflow.config
Jul-17 17:43:40.535 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/sme229/.nextflow/secrets/store.json
Jul-17 17:43:40.537 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@7b18658a] - activable => nextflow.secret.LocalSecretsProvider@7b18658a
Jul-17 17:43:40.545 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `singularity`
Jul-17 17:43:40.646 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [standard, conda, singularity, awsbatch, local]
Jul-17 17:43:40.667 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declaration
Jul-17 17:43:40.668 [main] DEBUG nextflow.cli.CmdRun - Launching `https://github.com/epi2me-labs/wf-single-cell` [desperate_cantor] DSL2 - revision: a25ec6cbf6 [master]
Jul-17 17:43:40.669 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jul-17 17:43:40.669 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Jul-17 17:43:40.708 [main] DEBUG nextflow.Session - Session UUID: 0a7313df-a924-40f4-8342-55427c0911c5
Jul-17 17:43:40.708 [main] DEBUG nextflow.Session - Run name: desperate_cantor
Jul-17 17:43:40.709 [main] DEBUG nextflow.Session - Executor pool size: 8
Jul-17 17:43:40.715 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Jul-17 17:43:40.719 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=24; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jul-17 17:43:40.746 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 24.04.2 build 5914
  Created: 29-05-2024 06:19 UTC (16:19 AEDT)
  System: Linux 5.14.21-150400.24.28-default
  Runtime: Groovy 4.0.21 on OpenJDK 64-Bit Server VM 11.0.21+0-suse-150000.3.107.1-x8664
  Encoding: UTF-8 (UTF-8)
  Process: 20275@c128 [10.141.201.128]
  CPUs: 8 - Mem: 256 GB (255.6 GB) - Swap: 64 GB (62.3 GB)
Jul-17 17:43:40.772 [main] DEBUG nextflow.Session - Work-dir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work [nfs]
Jul-17 17:43:40.794 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jul-17 17:43:40.802 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jul-17 17:43:40.864 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jul-17 17:43:40.873 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
Jul-17 17:43:40.989 [main] DEBUG nextflow.Session - Session start
Jul-17 17:43:40.991 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/output/execution/trace.txt
Jul-17 17:43:40.996 [main] DEBUG nextflow.Session - Using default localLib path: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/lib
Jul-17 17:43:41.000 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/lib
Jul-17 17:43:41.001 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/lib/nfcore_external_java_deps.jar
Jul-17 17:43:42.149 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jul-17 17:43:43.644 [main] WARN  nextflow.Nextflow - Found unexpected parameters:
* --kit_version: v3
* --kit_name: 3prime
Jul-17 17:43:43.645 [main] INFO  nextflow.Nextflow - - Ignore this warning: params.schema_ignore_params = "kit_version,kit_name" 
Jul-17 17:43:43.659 [main] INFO  nextflow.Nextflow - 
||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-single-cell v2.1.0-ga25ec6c
--------------------------------------------------------------------------------
Core Nextflow options
  revision       : master
  runName        : desperate_cantor
  containerEngine: singularity
  container      : [withLabel:singlecell:ontresearch/wf-single-cell:sha0fcdf10929fbef2d426bb985e16b81153a88c6f4, withLabel:wf_common:ontresearch/wf-common:sha338caea0a2532dc0ea8f46638ccc322bb8f9af48]
  launchDir      : /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024
  workDir        : /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work
  projectDir     : /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell
  userName       : sme229
  profile        : singularity
  configFiles    : /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/nextflow.config

Input Options
  fastq          : wf-single-cell-demo/chr17.fq.gz
  ref_genome_dir : wf-single-cell-demo
  expected_cells : 100

Advanced options
  umap_plot_genes: wf-single-cell-demo/umap_plot_genes.csv

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-single-cell for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-single-cell v2.1.0-ga25ec6c.
--------------------------------------------------------------------------------
Jul-17 17:43:44.961 [main] INFO  nextflow.Nextflow - Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
Jul-17 17:43:45.078 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `ingress,wf_common` for process with name fastcat
Jul-17 17:43:45.086 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.086 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.090 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Jul-17 17:43:45.094 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=8; memory=256 GB; capacity=8; pollInterval=100ms; dumpInterval=5m
Jul-17 17:43:45.096 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Jul-17 17:43:45.182 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name parse_kit_metadata
Jul-17 17:43:45.183 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.183 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.226 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:getVersions
Jul-17 17:43:45.227 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.227 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.229 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:getParams
Jul-17 17:43:45.230 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.230 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.244 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:preprocess:call_paftools
Jul-17 17:43:45.244 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.244 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.247 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:preprocess:get_chrom_sizes
Jul-17 17:43:45.248 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.248 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.252 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:preprocess:build_minimap_index
Jul-17 17:43:45.252 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.252 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.261 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:preprocess:call_adapter_scan
Jul-17 17:43:45.262 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.262 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.272 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:preprocess:summarize_adapter_table
Jul-17 17:43:45.272 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.272 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.278 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:split_gtf_by_chroms
Jul-17 17:43:45.279 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.279 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.287 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:generate_whitelist
Jul-17 17:43:45.288 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.288 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.295 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:assign_barcodes
Jul-17 17:43:45.296 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.296 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.300 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `wf_common` for process with name pipeline:process_bams:cat_tags_by_chrom
Jul-17 17:43:45.301 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.301 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.306 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `wf_common` for process with name pipeline:process_bams:merge_bams
Jul-17 17:43:45.307 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.307 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.318 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:stringtie
Jul-17 17:43:45.319 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.319 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.324 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:align_to_transcriptome
Jul-17 17:43:45.325 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.325 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.334 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:assign_features
Jul-17 17:43:45.335 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.335 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.342 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:create_matrix
Jul-17 17:43:45.343 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.343 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.355 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:process_matrix
Jul-17 17:43:45.356 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.356 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.363 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:merge_transcriptome
Jul-17 17:43:45.364 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.364 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.368 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:combine_final_tag_files
Jul-17 17:43:45.369 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.369 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.373 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:tag_bam
Jul-17 17:43:45.374 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.374 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.376 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:umi_gene_saturation
Jul-17 17:43:45.377 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.377 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.384 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:process_bams:pack_images
Jul-17 17:43:45.384 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.384 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.411 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:singlecell` matches labels `singlecell` for process with name pipeline:prepare_report_data
Jul-17 17:43:45.412 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.412 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.432 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `wf_common` for process with name pipeline:makeReport
Jul-17 17:43:45.433 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-17 17:43:45.433 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-17 17:43:45.437 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: move_or_compress_fq_file, pipeline:getParams, pipeline:process_bams:merge_bams, summarize_adapter_table, pipeline:preprocess:build_minimap_index, pipeline:process_bams:combine_final_tag_files, pipeline:process_bams:pack_images, checkBamHeaders, mergeBams, pipeline:process_bams:assign_barcodes, makeReport, get_chrom_sizes, pipeline:preprocess:summarize_adapter_table, getVersions, call_paftools, validateIndex, combine_final_tag_files, pack_images, merge_bams, process_matrix, bamstats, cat_tags_by_chrom, pipeline:process_bams:process_matrix, assign_features, fastcat, pipeline:prepare_report_data, parse_kit_metadata, generate_whitelist, pipeline:process_bams:stringtie, samtools_index, getParams, pipeline:process_bams:tag_bam, prepare_report_data, sortBam, validate_sample_sheet, pipeline:process_bams:cat_tags_by_chrom, create_matrix, pipeline:process_bams:umi_gene_saturation, pipeline:makeReport, pipeline:preprocess:call_adapter_scan, pipeline:process_bams:split_gtf_by_chroms, assign_barcodes, build_minimap_index, tag_bam, call_adapter_scan, umi_gene_saturation, pipeline:process_bams:generate_whitelist, pipeline:process_bams:align_to_transcriptome, stringtie, pipeline:process_bams:create_matrix, pipeline:process_bams:merge_transcriptome, pipeline:getVersions, merge_transcriptome, pipeline:preprocess:get_chrom_sizes, split_fq_file, pipeline:preprocess:call_paftools, pipeline:process_bams:assign_features, align_to_transcriptome, catSortBams, split_gtf_by_chroms
Jul-17 17:43:45.438 [main] DEBUG nextflow.Session - Igniting dataflow network (27)
Jul-17 17:43:45.438 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > fastcat
Jul-17 17:43:45.438 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > parse_kit_metadata
Jul-17 17:43:45.438 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:getVersions
Jul-17 17:43:45.438 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:getParams
Jul-17 17:43:45.438 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess:call_paftools
Jul-17 17:43:45.438 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess:get_chrom_sizes
Jul-17 17:43:45.439 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess:build_minimap_index
Jul-17 17:43:45.439 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess:call_adapter_scan
Jul-17 17:43:45.439 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:preprocess:summarize_adapter_table
Jul-17 17:43:45.445 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:split_gtf_by_chroms
Jul-17 17:43:45.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:generate_whitelist
Jul-17 17:43:45.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:assign_barcodes
Jul-17 17:43:45.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:cat_tags_by_chrom
Jul-17 17:43:45.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:merge_bams
Jul-17 17:43:45.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:stringtie
Jul-17 17:43:45.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:align_to_transcriptome
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:assign_features
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:create_matrix
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:process_matrix
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:merge_transcriptome
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:combine_final_tag_files
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:tag_bam
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:umi_gene_saturation
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:process_bams:pack_images
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:prepare_report_data
Jul-17 17:43:45.447 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > pipeline:makeReport
Jul-17 17:43:45.448 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
  Script_8d9f73166785a841: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/./lib/ingress.nf
  Script_e531f16a1e40dc47: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/main.nf
  Script_ec28993975b63c38: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/./subworkflows/preprocess.nf
  Script_0ff32ae3fcb77a5b: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/./subworkflows/process_bams.nf
Jul-17 17:43:45.448 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination 
Jul-17 17:43:45.448 [main] DEBUG nextflow.Session - Session await
Jul-17 17:43:45.503 [Actor Thread 17] DEBUG nextflow.container.SingularityCache - Singularity found local store for image=docker://ontresearch/wf-single-cell:sha0fcdf10929fbef2d426bb985e16b81153a88c6f4; path=/datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/singularity/ontresearch-wf-single-cell-sha0fcdf10929fbef2d426bb985e16b81153a88c6f4.img
Jul-17 17:43:45.506 [Actor Thread 22] DEBUG nextflow.container.SingularityCache - Singularity found local store for image=docker://ontresearch/wf-common:sha338caea0a2532dc0ea8f46638ccc322bb8f9af48; path=/datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/singularity/ontresearch-wf-common-sha338caea0a2532dc0ea8f46638ccc322bb8f9af48.img
Jul-17 17:43:45.580 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:43:45.581 [Task submitter] INFO  nextflow.Session - [11/e91ce2] Submitted process > pipeline:preprocess:build_minimap_index
Jul-17 17:43:49.707 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 6; name: pipeline:preprocess:build_minimap_index; status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/11/e91ce2834399f05fef801ba8442f92]
Jul-17 17:43:49.708 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'TaskFinalizer' minSize=10; maxSize=24; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jul-17 17:43:49.718 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:43:49.718 [Task submitter] INFO  nextflow.Session - [aa/c2a09b] Submitted process > pipeline:process_bams:split_gtf_by_chroms
Jul-17 17:43:49.726 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:43:49.726 [Task submitter] INFO  nextflow.Session - [4b/41f781] Submitted process > pipeline:getParams
Jul-17 17:43:49.743 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:43:49.744 [Task submitter] INFO  nextflow.Session - [da/73e562] Submitted process > fastcat (1)
Jul-17 17:43:49.751 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:43:49.752 [Task submitter] INFO  nextflow.Session - [31/821f56] Submitted process > pipeline:getVersions
Jul-17 17:43:49.761 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:43:49.761 [Task submitter] INFO  nextflow.Session - [5e/c0285e] Submitted process > pipeline:preprocess:get_chrom_sizes
Jul-17 17:43:50.126 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: pipeline:getParams; status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/4b/41f7818998dbce0a59ad11e635c715]
Jul-17 17:43:50.147 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:43:50.147 [Task submitter] INFO  nextflow.Session - [4f/af8c05] Submitted process > pipeline:preprocess:call_paftools
Jul-17 17:43:50.293 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 5; name: pipeline:preprocess:get_chrom_sizes; status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/5e/c0285e861b8a27ed7e88fa83e301f0]
Jul-17 17:43:50.933 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: pipeline:process_bams:split_gtf_by_chroms; status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/aa/c2a09b9459d5b432f6a39d87d35313]
Jul-17 17:43:51.206 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: pipeline:preprocess:call_paftools; status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/4f/af8c05aa8e823efa01932cd9fe65de]
Jul-17 17:43:57.774 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: pipeline:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/31/821f565cee0854d1c8d9d1bcd3506a]
Jul-17 17:43:59.929 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 7; name: fastcat (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/da/73e562f1153b5045decb8e3e2ba4b9]
Jul-17 17:43:59.975 [Actor Thread 10] DEBUG nextflow.sort.BigSort - Sort completed -- entries: 1; slices: 1; internal sort time: 0.0 s; external sort time: 0.009 s; total time: 0.009 s
Jul-17 17:44:00.031 [Actor Thread 10] DEBUG nextflow.file.FileCollector - Saved collect-files list to: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/collect-file/603348f8ea9d5800236691574bada1b3
Jul-17 17:44:00.040 [Actor Thread 10] DEBUG nextflow.file.FileCollector - Deleting file collector temp dir: /tmp/sme229.51310254/nxf-880416683842755020
Jul-17 17:44:00.064 [Actor Thread 3] DEBUG nextflow.util.HashBuilder - Hash asset file sha-256: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/data/OPTIONAL_FILE
Jul-17 17:44:00.067 [Actor Thread 3] DEBUG nextflow.util.HashBuilder - Hash asset file sha-256: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/kit_configs.csv
Jul-17 17:44:00.068 [Actor Thread 3] DEBUG nextflow.util.HashBuilder - Hash asset file sha-256: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue
Jul-17 17:44:00.110 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:44:00.110 [Task submitter] INFO  nextflow.Session - [e0/84dc78] Submitted process > parse_kit_metadata (1)
Jul-17 17:44:02.113 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 8; name: parse_kit_metadata (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/e0/84dc78d5e25efbfd0f5e7637351af4]
Jul-17 17:44:02.138 [Actor Thread 6] DEBUG nextflow.util.HashBuilder - Hash asset file sha-256: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/data/737K-august-2016.txt.gz
Jul-17 17:44:02.146 [Actor Thread 6] DEBUG nextflow.util.HashBuilder - Hash asset file sha-256: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/data/3M-february-2018.txt.gz
Jul-17 17:44:02.182 [Actor Thread 6] DEBUG nextflow.util.HashBuilder - Hash asset file sha-256: /home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/data/737K-arc-v1.txt.gz
Jul-17 17:44:02.206 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:44:02.206 [Task submitter] INFO  nextflow.Session - [dc/6e4a4b] Submitted process > pipeline:preprocess:call_adapter_scan (1)
Jul-17 17:46:19.801 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 9; name: pipeline:preprocess:call_adapter_scan (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/dc/6e4a4bb087f0bde043bb00a7e361ca]
Jul-17 17:46:19.824 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:46:19.824 [Task submitter] INFO  nextflow.Session - [ba/8792a9] Submitted process > pipeline:process_bams:merge_bams (1)
Jul-17 17:46:23.161 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 12; name: pipeline:process_bams:merge_bams (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/ba/8792a9980bbc52b85bcdf846998f62]
Jul-17 17:46:23.174 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:46:23.174 [Task submitter] INFO  nextflow.Session - [df/f3df01] Submitted process > pipeline:process_bams:generate_whitelist (1)
Jul-17 17:46:23.182 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:46:23.182 [Task submitter] INFO  nextflow.Session - [7d/b154fb] Submitted process > pipeline:preprocess:summarize_adapter_table (1)
Jul-17 17:46:25.480 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 10; name: pipeline:preprocess:summarize_adapter_table (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/7d/b154fbf843db01bc09dccd9228b9c5]
Jul-17 17:46:25.490 [TaskFinalizer-1] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'PublishDir' minSize=10; maxSize=24; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jul-17 17:46:27.299 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 11; name: pipeline:process_bams:generate_whitelist (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/df/f3df01445a541758350afc73fe8c0b]
Jul-17 17:46:27.310 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:46:27.310 [Task submitter] INFO  nextflow.Session - [ac/68ff60] Submitted process > pipeline:process_bams:stringtie (1)
Jul-17 17:46:46.578 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 13; name: pipeline:process_bams:stringtie (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/ac/68ff607384d7ad99475c2e569e7c92]
Jul-17 17:46:46.587 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:46:46.587 [Task submitter] INFO  nextflow.Session - [c3/68ad1b] Submitted process > pipeline:process_bams:assign_barcodes (1)
Jul-17 17:46:54.164 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 14; name: pipeline:process_bams:assign_barcodes (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/c3/68ad1b85817309d03eeb6de04f95d4]
Jul-17 17:46:54.174 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:46:54.174 [Task submitter] INFO  nextflow.Session - [22/de7881] Submitted process > pipeline:process_bams:align_to_transcriptome (1)
Jul-17 17:48:08.821 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 15; name: pipeline:process_bams:align_to_transcriptome (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/22/de7881abc00ec03b278d020e4caed1]
Jul-17 17:48:08.838 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:08.839 [Task submitter] INFO  nextflow.Session - [7f/95f393] Submitted process > pipeline:process_bams:cat_tags_by_chrom (1)
Jul-17 17:48:09.610 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 16; name: pipeline:process_bams:cat_tags_by_chrom (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/7f/95f3933643d836d4454cef1af23de4]
Jul-17 17:48:09.659 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:09.659 [Task submitter] INFO  nextflow.Session - [4b/c19ef9] Submitted process > pipeline:process_bams:assign_features (1)
Jul-17 17:48:28.677 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 17; name: pipeline:process_bams:assign_features (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/4b/c19ef9167b0b333755253d6f54e10b]
Jul-17 17:48:28.708 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:28.708 [Task submitter] INFO  nextflow.Session - [75/29dd2a] Submitted process > pipeline:process_bams:merge_transcriptome (1)
Jul-17 17:48:28.732 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:28.732 [Task submitter] INFO  nextflow.Session - [a4/45cb89] Submitted process > pipeline:process_bams:create_matrix (1)
Jul-17 17:48:29.565 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 19; name: pipeline:process_bams:merge_transcriptome (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/75/29dd2aac8b67dbf1124e1bd9251a8b]
Jul-17 17:48:45.281 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 18; name: pipeline:process_bams:create_matrix (1); status: RUNNING; exit: -; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/a4/45cb89617d15ad4d84fa939a786312]
Jul-17 17:48:53.605 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 18; name: pipeline:process_bams:create_matrix (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/a4/45cb89617d15ad4d84fa939a786312]
Jul-17 17:48:53.699 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:53.699 [Task submitter] INFO  nextflow.Session - [34/cc5ab7] Submitted process > pipeline:process_bams:combine_final_tag_files (1)
Jul-17 17:48:53.763 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:53.763 [Task submitter] INFO  nextflow.Session - [d2/d44852] Submitted process > pipeline:process_bams:process_matrix (2)
Jul-17 17:48:53.813 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:53.813 [Task submitter] INFO  nextflow.Session - [fe/4b2f7a] Submitted process > pipeline:process_bams:process_matrix (1)
Jul-17 17:48:53.871 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:48:53.871 [Task submitter] INFO  nextflow.Session - [b3/a410a5] Submitted process > pipeline:process_bams:tag_bam (1)
Jul-17 17:48:54.493 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 20; name: pipeline:process_bams:combine_final_tag_files (1); status: COMPLETED; exit: 0; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/34/cc5ab726824cbe5c980e14aa548c66]
Jul-17 17:48:54.504 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below
~> TaskHandler[id: 24; name: pipeline:process_bams:umi_gene_saturation (1); status: NEW; exit: -; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/02/454ca7f5072d4dfbc07a108b100848]
Jul-17 17:49:00.641 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 21; name: pipeline:process_bams:process_matrix (1); status: COMPLETED; exit: 1; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/fe/4b2f7a70c9a77aa92c625c2cfec2e2]
Jul-17 17:49:00.643 [TaskFinalizer-1] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=pipeline:process_bams:process_matrix (1); work-dir=/datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/fe/4b2f7a70c9a77aa92c625c2cfec2e2
  error [nextflow.exception.ProcessFailedException]: Process `pipeline:process_bams:process_matrix (1)` terminated with an error exit status (1)
Jul-17 17:49:00.653 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 22; name: pipeline:process_bams:process_matrix (2); status: COMPLETED; exit: 1; error: -; workDir: /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/d2/d44852c0c369512b9d3b71e8a02c7f]
Jul-17 17:49:00.656 [TaskFinalizer-1] ERROR nextflow.processor.TaskProcessor - Error executing process > 'pipeline:process_bams:process_matrix (1)'

Caused by:
  Process `pipeline:process_bams:process_matrix (1)` terminated with an error exit status (1)

Command executed:

  export NUMBA_NUM_THREADS=1
  workflow-glue process_matrix         inputs/matrix*.hdf         --feature transcript         --raw transcript_raw_feature_bc_matrix         --processed transcript_processed_feature_bc_matrix         --per_cell_mito transcript.expression.mito-per-cell.tsv         --per_cell_expr transcript.expression.mean-per-cell.tsv         --umap_tsv transcript.expression.umap.tsv         --enable_filtering         --min_features 200         --min_cells 3         --max_mito 20         --mito_prefixes MT-         --norm_count 10000         --enable_umap         --replicates 3

Command exit status:
  1

Command output:
  (empty)

Command error:
  [17:48:54 - workflow_glue] Bootstrapping CLI.
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  Traceback (most recent call last):
    File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in <module>
      cli()
    File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 66, in cli
      components = get_components(allowed_components=[sys.argv[1]])
    File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 29, in get_components
      mod = importlib.import_module(f"{_package_name}.{name}")
    File "/home/epi2melabs/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
    File "<frozen importlib._bootstrap>", line 991, in _find_and_load
    File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 843, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/home/sme229/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/process_matrix.py", line 8, in <module>
      import umap
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/__init__.py", line 2, in <module>
      from .umap_ import UMAP
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py", line 41, in <module>
      from umap.layouts import (
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py", line 40, in <module>
      def rdist(x, y):
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/decorators.py", line 234, in wrapper
      disp.enable_caching()
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/dispatcher.py", line 863, in enable_caching
      self._cache = FunctionCache(self.py_func)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 601, in __init__
      self._impl = self._impl_class(py_func)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 337, in __init__
      raise RuntimeError("cannot cache function %r: no locator available "
  RuntimeError: cannot cache function 'rdist': no locator available for file '/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py'

Work dir:
  /datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/fe/4b2f7a70c9a77aa92c625c2cfec2e2

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
Jul-17 17:49:00.658 [TaskFinalizer-2] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=pipeline:process_bams:process_matrix (2); work-dir=/datasets/work/hb-rabbit-gbc/work/Maria_Jenkel_ONT_Data_04-07-2024/work/d2/d44852c0c369512b9d3b71e8a02c7f
  error [nextflow.exception.ProcessFailedException]: Process `pipeline:process_bams:process_matrix (2)` terminated with an error exit status (1)
Jul-17 17:49:00.660 [TaskFinalizer-1] DEBUG nextflow.Session - Session aborted -- Cause: Process `pipeline:process_bams:process_matrix (1)` terminated with an error exit status (1)
Jul-17 17:49:00.661 [Actor Thread 23] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
Jul-17 17:49:00.662 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-17 17:49:00.662 [Task submitter] INFO  nextflow.Session - [02/454ca7] Submitted process > pipeline:process_bams:umi_gene_saturation (1)
Jul-17 17:49:00.720 [TaskFinalizer-1] DEBUG nextflow.Session - The following nodes are still active:
[process] pipeline:process_bams:pack_images
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

Jul-17 17:49:00.747 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Jul-17 17:49:01.650 [main] DEBUG nextflow.Session - Session await > all processes finished
Jul-17 17:49:01.651 [main] DEBUG nextflow.Session - Session await > all barriers passed
Jul-17 17:49:01.963 [main] WARN  n.processor.TaskPollingMonitor - Killing running tasks (2)
Jul-17 17:49:01.977 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=20; failedCount=2; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=1; runningCount=-1; retriesCount=0; abortedCount=2; succeedDuration=33m 32s; failedDuration=13.5s; cachedDuration=0ms;loadCpus=-4; loadMemory=0; peakRunning=5; peakCpus=8; peakMemory=49 GB; ]
Jul-17 17:49:01.977 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Jul-17 17:49:01.979 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Jul-17 17:49:02.371 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Jul-17 17:49:02.416 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Jul-17 17:49:02.441 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
nrhorner commented 4 months ago

Hi @sme229

This appears to be related to wf-single-cell. Could you repost there please?

sme229 commented 4 months ago

I have resolved the issue