Open zhujack opened 3 weeks ago
nrhorner
commented
2 weeks ago Hi @zhujack could you add the following to a file (numba.config)
env {
NUMBA_CACHE_DIR = "${launchDir}/numba_cache"
}And repeat the command and include -resume -c numba.config in your command.
e-hutchins
commented
1 week ago Hi @zhujack I am having this same issue with the demo data.
I tried your suggestion @nrhorner and I get the same error above. I am also deploying on a SLURM cluster.
Andrey-Tyshevich
commented
2 days ago Hi @zhujack @e-hutchins I encountered the same error using slurm profile with singularity image. The solution is to assign NUMBA_CACHE_DIR the destination the nextflow processes can write to, e.g. you can specify:
singularity {
runOptions = '-B /your_real_path:/your_real_path'
}
env {
NUMBA_CACHE_DIR = "/your_real_path/numba_cache"
}@Andrey-Tyshevich that did the trick, got the test data to run. Thank you!
Operating System
Other Linux (please specify below)
Other Linux
Oracle Linux 8
Workflow Version
v2.3.0
Workflow Execution
Command line (Cluster)
Other workflow execution
No response
EPI2ME Version
No response
CLI command run
nextflow run epi2me-labs/wf-single-cell \ --expected_cells 100 \ --fastq 'wf-single-cell-demo/chr17.fq.gz' \ --kit '3prime:v3' \ --ref_genome_dir 'wf-single-cell-demo' \ --genes_of_interest 'wf-single-cell-demo/umap_plot_genes.csv' \ --threads $SLURM_CPUS_PER_TASK \ -profile singularity
Workflow Execution - CLI Execution Profile
singularity
What happened?
ERROR ~ Error executing process > 'pipeline:process_bams:process_matrix (1)'
Caused by: Process
pipeline:process_bams:process_matrix (1)terminated with an error exit status (1)Command executed:
export NUMBA_NUM_THREADS=1 workflow-glue process_matrix inputs/matrix*.hdf --feature gene --raw gene_raw_feature_bc_matrix --processed gene_processed_feature_bc_matrix --per_cell_mito gene.expression.mito-per-cell.tsv --per_cell_expr gene.expression.mean-per-cell.tsv --umap_tsv gene.expression.umap.tsv --enable_filtering --min_features 200 --min_cells 3 --max_mito 20 --mito_prefixes MT- --norm_count 10000 --enable_umap --replicates 3
Command exit status: 1
Command output: (empty)
Command error: [08:23:08 - workflow_glue] Bootstrapping CLI. /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() Traceback (most recent call last): File "/data/CCRBioinfo/zhujack/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in
cli()
File "/data/CCRBioinfo/zhujack/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/init.py", line 66, in cli
components = get_components(allowed_components=[sys.argv[1]])
File "/data/CCRBioinfo/zhujack/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/init.py", line 29, in get_components
mod = importlib.import_module(f"{_package_name}.{name}")
File "/home/epi2melabs/conda/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 843, in exec_module
File "", line 219, in _call_with_frames_removed
File "/data/CCRBioinfo/zhujack/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/processmatrix.py", line 8, in
import umap
File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/init.py", line 2, in
from .umap import UMAP
File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py", line 41, in
from umap.layouts import (
File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py", line 40, in
def rdist(x, y):
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/decorators.py", line 234, in wrapper
disp.enable_caching()
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/dispatcher.py", line 863, in enable_caching
self._cache = FunctionCache(self.py_func)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 601, in init
self._impl = self._impl_class(py_func)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 337, in init
raise RuntimeError("cannot cache function %r: no locator available "
RuntimeError: cannot cache function 'rdist': no locator available for file '/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py'
Work dir: /vf/users/CCRBioinfo/zhujack/projects/tests/tool_test/ONT/wf-single-cell/work/10/65ac3d3e20d8ac6bf1968342c61c20
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named
.command.sh-- Check '.nextflow.log' file for details
Relevant log output
Application activity log entry
No response
Were you able to successfully run the latest version of the workflow with the demo data?
no
Other demo data information
No response